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Executive Summary 



The No Child Left Behind Act of 2001 (PL 107-110) established the Reading First Program 
(Title I, Part B, Subpart 1), a major federal initiative designed to help ensure that all children can 
read at or above grade level by the end of third grade. Reading First (RF) is predicated on 
scientifically researched findings that high-quality reading instruction in the primary grades 
significantly reduces the number of students who experience reading difficulties in later years. 

A. Key Provisions of the Reading First Program 

The Reading First program’s overarching goal is to improve the quality of reading instruction — 
and thereby improve the reading skills and achievement of children in the primary grades — by 
providing substantial resources at both the state and local levels. The intent is to ensure that 
teachers in kindergarten through third grade use reading programs and materials that are 
research-based. Additionally, Reading First intends to increase access to and the quality of 
professional development for ah teachers of these grades, including special education teachers, to 
ensure that they have the necessary skills to teach these researched-based reading programs 
effectively. An important provision of the RF legislation is that professional development be 
made available to all schools, not only schools that received RF funding. A third emphasis is on 
using assessments, both to monitor progress and to identify students’ reading problems early on. 
Reading First is intended to help prepare classroom teachers to screen for, identify, and 
overcome barriers to students’ ability to read at grade level by the end of third grade. More 
specifically, the programs and the professional development provided to school staff must use 
reading instructional methods and materials that incorporate the five essential elements of 
effective primary-grade reading instruction, as specified in the legislation: 1) phonemic 
awareness; 2) decoding; 3) vocabulary development; 4) reading fluency, including oral reading 
skills; and 5) reading comprehension strategies. 

All 50 states and other jurisdictions* have been awarded Reading First grants. To date (April 
2006), states have awarded subgrants to approximately 1,550 local school districts and, in turn, 
these districts have provided funds to approximately 5,200 schools nationwide. Because grants to 
states were awarded over an extended time period and states differed in the amount of time they 
allotted to their competitive subgrant processes, districts and schools are at various stages of 
implementing their Reading First programs. 

B. Overview of the Evaluation 

The enabling legislation for RF requires the U.S. Department of Education (ED) to contract with 
an outside entity to evaluate the program’s implementation (Section 1205). To meet this 



* State Education Agencies (SEAs) were eligible to apply for RE grants. Other jurisdictions eligible include 
District of Columbia, the schools of the Bureau of Indian Affairs, and American Samoa. Guam and Northern 
Mariana Islands received grants through the consolidated grants to insular areas. 
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requirement, the Department contracted with Abt Associates in October 2003 to design and 
conduct the Reading First Implementation Evaluation, which addresses the following questions: 

1. How is the Reading First program implemented in districts and schools? 

2. How does reading instruction differ between Reading First schools and non-RF Title I 
schools? 

3. How does reading instruction differ between Reading First schools and non-RF Title I 
schools as RF schools’ implementation efforts mature over time? 

4. Does student achievement improve in schools with Reading First funds? 

5. Is there any relationship between how schools implement Reading First and changes 
in reading achievement? 

The five-year study has produced this interim report based on data collected during the 2004-05 
school year as well as analyses of extant data sources; it will also produce a final report in 2007 
based on data from the 2006-07 school year and updated extant data. This interim report 
addresses questions 1 and 2. Question 3 requires an analysis of longitudinal data and will be 
addressed upon completion of the second wave of data collection in 2007, as will questions 4 and 
5. Below we summarize key findings from the evaluation, using the following data sources: 

• Surveys completed in spring 2005 by 6,185 K-3 teachers, 1,574 principals, and 1,318 
reading coaches in nationally representative samples of 1,092 Reading First schools 
and 541 non-RF Title I schools; 

• Interviews with Reading First state coordinators, and reviews of states’ applications 
for RF awards; 

• The Reading First Awards Database that lists all RF districts and schools as well as 
their baseline measures of K-3 reading performance and poverty rates (as measured 
by percent of students receiving free and reduced-price lunches); and 

• ED’s School-Fevel State Assessment Score Database (SFAD) that provided measures 
of reading achievement and poverty for all school districts nationwide. 

The non-RF Title I school sample was constructed purposefully to provide a context for 
understanding how reading programs in a sample of Reading First schools differ from those in 
schools serving similar populations of students. The non-RF sample includes only Title I 
schoolwide project (SWP) schools with at least 40 percent of the students eligible for free or 
reduced price lunches, which is comparable to the RF school population. The two groups of 
schools are demographically similar in staff experience, attendance rates, mobility, and stability 
of enrollment. RF schools are, however, on average, larger than the Title I schools, and have 
larger proportions of K-3 students reading below grade level. 

^ The most rigorous design option available for this evaluation would have been to identify a group of non-RF 
schools matched to RF schools on key demographic and achievement characteristics to minimize differences 
between RF and non-RF schools and thereby approximate a random assignment experiment. However, because 
RF schools, by definition, are among the lowest performing schools in their respective districts, matched 
comparison schools could include better performing schools. Also, RF schools could likely differ from similar 
non-funded schools because, often, they had to demonstrate motivation, and this factor could influence any 
observed instructional differences between RF and comparison schools. 
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We can make comparisons between RF and non-RF Title I samples, but because the two samples 
are not matched they cannot be assumed to be equivalent. Thus, the differences between the 
groups discussed in this report cannot be attributed to the Reading First program. 



C. Results of the Reading First Implementation Evaluation 



Key Finding 

Reading First schools appear to be implementing the major elements of the program as 
intended by the legislation, such as providing scientifically based reading instruction in grades 
K-3, increased amounts of time for reading instruction, interventions for struggling readers, 
wider use of classroom-based reading assessments, and more professional development 
activities. 



External Resources to Support Reading Programs 

Key Findings 

Reading First schools received both financial and nonfinancial support from a variety of 
external sources. During the 2004-05 school year, the median annual amount of funds RF 
schools received to implement their reading program was $138,000. In addition, Reading 
First schools have multiple external resources, in addition to RF funds, to support the 
implementation of their reading programs. 



Most RF schools also received Title I funds (91 percent) and district funds (79 percent) to 
support their reading programs. Exhibit E-1 illustrates that beyond financial support, according 
to principals, RE schools received substantially more external assistance than did Title I schools 
on selecting instructional programs (76 percent vs. 56 percent), diagnosing needs of struggling 
readers (70 percent vs. 50 percent), conducting demonstration lessons (71 percent vs. 48 
percent), and reviewing the effectiveness of reading programs (71 percent vs. 47 percent). 
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Exhibit E-1 

Nonfinanciai Externai Assistance for K-3 Reading Program Activities in Reading First 




Conducting Conducting Diagnosing needs Seiecting Providing TA in Reviewing 
Demonstration Ciassroom of struggling instructional implementing reading program 
lessons observation readers programs core reading effectiveness 

/materials programs 

Source: Principal Survey, Question B7. 

Exhibit reads: 7 1 percent of principals in RF schools reported receiving external assistance in conducting 
demonstration lessons, compared to 48 percent of principals in Title I schools. This difference is statistically 
significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 
Title I schools. 

TA stands for technical assistance. 



Reading Instruction in K-3 Ciassrooms 



Key Finding 

Classroom reading instruction in RF schools is significantly more likely to adhere to 
the RF legislation than that in Title I schools. Reading instruction encompasses the 
amount of instructional time, use of appropriate reading materials, and implementing 
reading activities and strategies supported by scientifically based reading research. 



Instructional Time 

The Guidance for the Reading First Program states that schools “should consider the allocation 
of time, including a protected, uninterrupted block of time for reading instruction of more than 
90 minutes per day.” Significantly more RF schools than Title I schools reported having a 
reading block for each of grades K through 3 (grades 1-3; 98 percent vs. 92 percent, 
kindergarten; 98 percent vs. 88 percent. Exhibit E-2). Teachers in Reading Eirst schools reported, 
on average, that they spent significantly more time on reading than did teachers in non-RE Title I 
schools — a difference of about 19 minutes per day, or almost 100 minutes per week. Teachers in 
newly funded RE schools were also significantly more likely than teachers in non-RE Title I 
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schools to report that they had increased the amount of time spent on reading from the 2003-04 
to the 2004-05 school years (61 percent vs. 35 percent).^ 



Exhibit E-2 

Scheduied Reading Biocks in Reading First and Titie i Schoois, 2004-05 Schooi Year 




Percent of Schools with Reading Block 



Source: Reading Coach Survey and Principal Survey, Questions D2 and D3. 

Exhibit reads: 98 percent of RF schools reported having a scheduled block at grades 1-3, compared with 92 percent of 
Title I schools (p < .05). 

Note: The stars (* *) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I schools. 



Staff in RF schools also reported (using a five-point seale"^) having signifieantly more time set 
aside than staff in non-RF Title I sehools to use assessment data to plan instruetion (3.88 vs. 
3.38); to observe reading instruction in other classrooms (2.21 vs. 1.96); to collaborate on 
reading lesson planning and instruetion (4.23 vs. 3.88); and to reeeive instruetion themselves 
from a reading coach (once a month on average for RF schools, vs. four or fewer times per year 
in non-RF Title I schools). 

Instructional Materials 

Reading First sehools reported that they have made substantial changes to their reading programs 
since they received their RF funds in the 2004-05 sehool year (Exhibit E-3). Newly funded RE 
sehools were signifieantly more likely than Title I sehools to have adopted a new eore reading 
program (39 percent vs. 16 percent), to have added new intervention programs for struggling 
readers (74 pereent vs. 43 pereent), to have added new supplementary materials (69 percent vs. 

58 pereent), and to have adopted new materials for English Eanguage Eearners (43 percent vs. 29 
pereent). 



^ For this comparison, we used the newly funded RF schools since the mature RF schools may have already 

increased their instructional time in their first year of implementation, 2002-03. That said, about 45 percent of 
the mature RF schools reported increasing their instructional time in the 2004-05 school year. 

* The scale represents how often schools reported time being set aside during the school year: 1 = Not at all, 2 = 
1—4 times, 3 = 5-8 times, 4 = Once a month, and 5 = Once a week or more. 
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Exhibit E-3 

Changes to Reading Program Materiais for Newiy Funded Reading First Schoois and Titie i 




Adopted a new Added a new Added new Added new Adopted new 

core reading intervention suppiementary materiais for ELLs reading 

program program for materials assessments 

struggling 
readers 

Source: Reading Coach Survey Question C3 and Principal Survey, Question D7. 

Exhibit reads: 39 percent of the newly funded Reading First schools reported adopting a new core reading program at the 
beginning of the 2004—05 school year, compared with 16 percent of Title I schools. This difference is statistically significant (p 
< .05). 

Note: The stars (* *) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I schools. 



Staff in mature RF schools were significantly more likely than staff in non-RF Title I schools to 
rate (on a five-point scale) the following statements about their schools’ reading programs more 
positively: 

• The core reading program is aligned with scientifically based reading research 
(SBRR) (4.63 vs. 4.29). 

• Reading intervention materials are aligned with SBRR (4.38 vs. 4.22). 

• High-quality instructional materials are available (4.39 vs. 4.01). 



Despite evidence of greater alignment with SBRR, staff in mature RF schools recognized their 
lack of experience working with some materials. They were significantly more likely than staff 
in non-RF Title I schools to rate (also on a five-point scale) the following statements more 
negatively: 1) Teachers are experienced with supplemental reading materials (3.29 vs. 3.58); and 
2) Teachers are experienced with reading intervention materials (3.19 vs. 3.39). This may reflect 
the fact that many of these materials are new in RF schools. 

Instructional Strategies 

Reading instructional activities and strategies in RF schools appear to be aligned with the tenets 
of the Reading First program. Differences in instructional environments between mature RF and 
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non-RF Title I schools were manifested in a variety of ways.^ RF teachers across all grades (K- 
3) rated as central a greater proportion of SBRR-aligned practices than did teachers in Title I 
schools: These statistically significant differences were modest in size (e.g., second grade: 76 
percent vs. 72 percent), although the difference for kindergarten teachers was somewhat larger 
(77 percent vs. 68 percent). 

When asked specifically about the centrality of instruction in the five dimensions of reading, 
similar patterns emerge. There are differences that are modest in magnitude. Kindergarten and 
first-grade teachers rated as central to their instruction a significantly higher proportion of 
scientifically based practices in teaching phonemic awareness and decoding than did teachers in 
Title I schools (kindergarten: 91 percent vs. 86 percent; first grade: 85 percent vs. 81 percent). 
RF third-grade teachers rated as central to their instruction a significantly higher proportion of 
scientifically based practices in teaching vocabulary and fluency than did their Title I 
counterparts (vocabulary, 75 percent vs. 70 percent; fluency. 56 percent vs. 47 percent). There 
were no differences between RF and Title I teachers in their centrality ratings related to teaching 
comprehension .skills. 

Interventions for Struggling Readers 



Key Findings 

RF teachers in three grades (kindergarten, second, and third) were significantly more likely 
than their counterparts in Title I schools to place their struggling students in intervention 
programs. 

Based on principal reports, in both RF and Title I schools, there was no time delay between 
identifying students who need interventions and the provision of services to those students. RF and 
Title I schools were also similar with respect to planning and coordinating instruction for ELL 
students. 

Although significantly more RL teachers reported receiving professional development in helping 
struggling readers than did Title I teachers, teachers in both RL and Title I schools recognized the 
challenge of providing effective instruction to struggling readers; 80 percent of teachers in both 
groups reported that they need additional professional development on this topic. 



Exhibit E-4 indicates that RL schools were more likely to rely on progress monitoring (98 
percent vs. 90 percent) and reading coach recommendations to identify struggling readers (92 
percent and 55 percent) than were Title I schools. Although there were no differences in 



We constructed six composites to summarize teachers’ ratings of the centrality of a series of instructional 
activities associated with the following reading dimensions and other instructional features: 1) phonemic 
awareness and decoding; 2) vocabulary; 3) comprehension; 4) fluency; 5) use of scientifically based 
instructional strategies and materials; and 6) negative alignment with scientifically based reading research. (See 
Appendix D for a list the specific items included in each composite.) Scores were computed for each composite 
based on the percentage of instructional activities specified in that composite that a teacher rated as “central to 
their instruction.’’ 
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principal reports of the use of diagnostic tests, RF teachers across all grades were significantly 
more likely to rely on diagnostic assessments to determine their struggling readers’ core deficits 
than were teachers in Title I schools (74 percent vs. 64 percent). 



Exhibit E-4 



Types of Assessments Reported by Principais That Were Used to identify 
Students for Reading interventions, in RF and Titie i Schoois, 2004-05 
Schooi Year 





Reading First 
Schools 


Titie 1 
Schools 


Type of Test 


Percent 


Percent 


Progress monitoring tests 


98%* 


90% 


Tests from core reading program 


95 


92 


Diagnostic tests 


91 


91 


Reading coach recommendation^ 


92* 


55 


Standardized achievement tests 


88 


88 


Screening tests 


87* 


82 



Source: Principal Survey, Question E2. 

Exhibit reads: Reading Eirst schools are more likely to use progress monitoring tests to identify 
students for reading interventions than are Title I schools (98 percent vs. 90 percent. This difference 
is statistically significant (p < .05). 

1 . Only schools that have reading coaches are included in the analysis of this item. 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) 
between RE and Title I schools. 



Despite these differences in the identification of struggling readers, there were no substantive or 
statistically significant differences between RF and Title I schools in terms of availability of 
intervention services for struggling readers; about 80 percent of both RF and Title I principals 
reported that reading intervention services are available when needed. Nor did the average 
waiting time for students’ receipt of services differ significantly between RF and Title I schools; 
two-thirds of the principals reported no wait time for students in need of intervention. On 
average, identified students received services within approximately one week. 

RF and Title I schools are more similar than different with respect to coordinating instruction for 
ELL students. There were no significant differences between RL and Title I teachers in reported 
time set aside to coordinate instruction with ELL staff; in fact, more than one-third of teachers 
who have ELL students in their classrooms (in both groups of schools) reported that no such time 
is specifically set aside. Only about 10 percent of teachers reported that they had weekly 
meetings with ELL staff to coordinate reading instruction for their struggling ELL students. It is 
important to note here, however, that some schools may not have any ELL teachers on staff. 

RL and Title I schools do differ in the provision of special education services to struggling 
readers. Title I schools were significantly more likely than RL schools to have a certified special 
education teacher provide recommendations to plan instruction for struggling readers (83 percent 
vs. 72 percent). RL teachers in kindergarten, first, and second grades were significantly more 
likely than Title I teachers to report that time is not set aside for coordination of the reading 
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instruction provided to their special education students (kindergarten: 51 percent vs. 44 percent; 
first grade: 42 percent vs. 35 percent; second grade: 37 percent vs. 30 percent). 

RF teachers were more likely to report increased amounts of time for struggling readers to 
practice skills in several dimensions of reading. RF teachers in three of four grades reported 
providing practice in significantly more reading dimensions than did teachers in Title I schools 
(kindergarten: 2.75 vs. 2.66; second grade: 2.83 vs. 2.78; third grade: 2.71 vs. 2.45). These 
differences, while statistically significant, are substantively quite small. RF third-grade teachers 
were significantly more likely than Title I teachers to provide struggling readers with extra 
practice in decoding (92 percent vs. 82 percent) and fluency (97 percent vs. 90 percent); there 
were no significant differences for the other grades. 

In terms of materials, RF teachers were also more likely to draw upon resources external to the 
core reading program to support struggling readers. Kindergarten and third-grade teachers in RF 
schools were significantly more likely than their counterparts in Title I schools to use materials 
that supplement the core reading program (kindergarten teachers: 70 percent vs. 62 percent; 
third-grade teachers 74 percent vs. 66 percent). There were no significant differences in first or 
second grade. RF teachers in first and second grade were significantly more likely than teachers 
in Title I schools to provide in-class help in reading for ELL students (66 percent vs. 55 percent) 
Kindergarten and third grade were not significantly different. 

Assessment 



Key Findings 

Assessment plays an important role in reading programs in both RF and non-RF Title I 
schools. 

Reading First schools received more outside assistance in selecting assessments than Title I 
schools. At the same time, staff in both RF and Title I reported that the district is largely 
responsible for the selection and interpretation of reading assessments. 

There were some differences in the types of assessments teachers in Reading First schools and 
teachers in non-RF Title I schools found useful. RF teachers were more likely to identify 
assessments from their core or supplementary reading programs as useful than are teachers in 
Title I schools. In contrast. Title I teachers were more likely to report that informal assessments 
are useful than are RF teachers. 

Teachers in RF schools were more likely to report applying assessment results for varied 
instructional purposes (e.g., for planning grouping, progress monitoring and identifying 
struggling readers) than their Title I counterparts. 



Executive Summary 



9 






Selection and Interpretation of Reading Assessments 

Reading First schools received significantly more outside assistance (from district, state, 
publisher, university expert, etc.) than did non-RF Title I schools in selecting assessment 
instruments for their K-3 reading program (76 percent vs. 56 percent) and interpreting 
assessment results (82 percent vs.70 percent). 

Overall, there were many similarities in RF and non-RF Title I principals’ reports of who is 
responsible for the selection and interpretation of reading assessments. More than three-quarters 
of both RF and non-RF Title I principals identified the district as responsible for selecting 
assessments. There were, however, some differences. Half of RF principals (51 percent) also 
reported state responsibility for selecting assessments, compared to 31 percent of non-RF Title I 
principals. In 54 percent of non-RF Title I schools, the principal also held this responsibility, 
compared to only 40 percent of RF principals. 

Nearly all, about 90 percent, Reading First and non-RF Title I principals reported that they, as 
principals, were responsible for interpreting assessment results. In 93 percent of RF schools, the 
reading coach also assumed responsibility for this task, compared to only half of Title I 
principals (52 percent). This significant difference reflects the fact that, in contrast to Reading 
First schools. Title I schools were much less likely to have a designated reading coach. 

Eighty-four percent of RF teachers reported that they had regularly scheduled, formal time set 
aside to use assessment data to plan instruction, compared with 74 percent of non-RF Title I 
teachers. 

Types of Reading Assessments Teachers Find Useful 

Most teachers in both RF and Title I schools named at least one assessment that they found 
useful in placing or grouping students (90 percent), determining student mastery of skills (89 
percent), and identifying the core deficits of struggling students (85 percent). 

In general, RF teachers were significantly more likely to identify formal assessments as useful 
than Title I teachers. In contrast. Title I teachers were more likely to report the usefulness of 
informal assessments than were RF teachers. Across assessment purposes, Reading First teachers 
were more likely than their Title I counterparts to identify assessments from the core or 
supplementary reading program (e.g., for determining student mastery of skills, 50 percent vs. 38 
percent) or standardized tests (e.g., for placing or grouping students, 50 percent vs. 44 percent), 
such as the DIB ELS (e.g., for identifying the core deficits of struggling students, 33 percent vs. 
10 percent). 

In contrast, across the three uses of assessments. Title I teachers were significantly more likely to 
identify informal assessments as useful than were Reading Eirst teachers (e.g., for determining 
student mastery of skills, 38 percent vs. 28 percent). Specifically, Title I teachers were more 
likely to report classroom-based assessments (e.g. for identifying the core deficits of struggling 
students, 24 percent vs. 17 percent) and running records or miscue analysis (e.g., for placing or 
grouping students, 19 percent vs. 10 percent) as useful than Reading Eirst teachers. 
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Most K-3 teachers in both Reading First and Title I schools named at least one assessment that 
they found useful in placing or grouping students (90 percent), determining student mastery of 
skills (89 percent), and identifying the core deficits of struggling students (85 percent). 

However, a significantly greater percentage of RF teachers reported that they use assessment 
results to organize instructional groups (83 percent vs. 73 percent), to determine progress on 
skills (85 percent vs. 78 percent), and to identify students who need reading intervention services 
(75 percent vs. 65 percent) than did Title I teachers. 

Oversight and Classroom Support Activities 



Key Findings 

Principals in Reading First schools were significantly more likely to report having a 
reading coach than were principals of non-RF Title I schools.® 

Coaches in RF schools were significantly more likely to provide teachers with various 
supports for their reading instruction than were coaches in non-RF Title I schools. 



Reading First schools were significantly more likely to have a reading coach (98 percent vs. 60 
percent) than were non-RF Title I schools, as reported by the principal, reflecting the fact that 
nearly all states required RF schools (but not non-RF Title I schools) to have a reading coach. 

Of the Title I schools that reported having reading coaches, 88 percent have a coach that is doing 

•j 

the central activities of a reading coach. In Reading First schools, reading coaches are intended 
to work primarily with teachers rather than directly with students. It is therefore noteworthy that 
a significantly smaller percentage of reading coaches in RF schools reported that providing 
direct reading instruction to students is absolutely central to their work: 29 percent, compared to 
53 percent in non-RF Title I schools (Exhibit E-5). 

Reading coaches in mature Reading Eirst schools were significantly more likely than their non- 
RE Title I counterparts to rate the following teacher support activities as absolutely central to 
their work: providing training or professional development (95 percent vs. 87 percent), coaching 
staff on a range of topics (92 percent vs. 83 percent), organizing professional development (87 
percent vs. 67 percent), and facilitating grade-level meetings (67 percent vs. 47 percent). 



® Most states require RF schools to employ reading coaches to support teachers’ reading instruction; this is not 
the case for non-RF Title 1 schools. 

^ Because respondents from non-RF Title 1 schools who completed the Reading Coach Survey reported a 
multiplicity of job titles, we used their responses to two survey questions to determine their inclusion in the 
comparison group of reading coaches; How central is each of the following activities? 1) “Coaches staff on a 
range of topics”; and 2) “Organizes professional development for K-3 teachers.” Respondents who answered a 
3 (“somewhat central”) or above (on a 5-point scale) for at least one of these two items were included in the 
comparison group of reading coaches from Title 1 schools for these sets of analyses regarding reading coach 
responsibilities. As a result, 34 reading coach respondents from Title schools, the equivalent of 940 weighted 
respondents, were excluded from these analyses. 
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Reading coaches in mature Reading First schools were significantly more likely than their non- 
RF Title I counterparts to report that the following administrative support activities were 
absolutely central to their work: compiling reading assessment data (88 percent vs. 67 percent) 
and administering or coordinating reading assessments (86 percent vs. 67 percent). About 75 
percent of both mature RF and non-RF Title I schools reported participating in school leadership 
team meetings, and about 70 percent of both types of schools reported managing reading 
instructional materials, as absolutely central to their work (with no significant differences). 
Significantly more reading coaches in mature RF schools, compared with coaches in non-RF 
Title I schools, characterized participating in professional development (97 percent vs. 86 
percent) as absolutely central to their work. 



Exhibit E-5 



Reading Coaches’ Ratings of the importance of Various Support Activities in Reading 
First and Titie i Schoois, 2004-05 Schooi Year 

Mature 

Reading First Title I 
Schools Schools 



Activity Rated as 

Reading Activity/ “Absolutely Central” 

Centrality of the Activity Percent Percent 



Teacher Support: 


Facilitate grade-level meetings 


67%* 


47% 


Coach staff on a range of topics 


92* 


83 


Provide direct reading instruction to students 


29 


53* 


Organize professional development for K-3 teachers 


67* 


47 


Provide training/professional development in reading 
materials, strategies, and assessments 


95* 


87 


Give demonstration lessons with core/supplemental 
materials 


79* 


70 


Observe and provide feedback to teachers 


84* 


69 


Assist teachers in forming instructional groups 


85* 


68 


Help teachers design strategies for struggling readers 


90 


86 


Give demonstrations on assessment administration/scoring 


71* 


56 


Administrative Support 


Administer reading assessments 


86* 


67 


Participate in school leadership team meetings 


76 


75 


Compile reading assessment data 


88* 


67 



Source: Reading Coach Survey, Question B4. 

Exhibit reads: 67 percent of reading coaches in RF schools rate facilitating grade-level meeting as “central” to 
their work, compared with 47 percent of reading coaches in Title I schools. This difference is statistically 
significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between reading 
coaches in RF schools and reading coaches in Title I schools. 

The responses of 34 reading coaches (940 weighted) from non-RF Title I schools from this analysis because, 
based on their survey responses, they did not appear to meet the definition of ’’reading coach.” 



12 



Executive Summary 










Professional Development 



Key Findings 

RF staff received significantly more professional development than did Title I staff. RF 

teachers were more likely to have received professional development in the five dimensions of 
reading instruction as well as in overall teaching strategies. Indeed, RF teachers reported 
feeling better prepared to teach the five dimensions of reading than their Title I counterparts. 

Activities attended by RF teachers were more likely to have structural attributes conducive to a 
successful experience, such as incentives and follow-up activities, than those attended by 
teachers in non-RF Title I teachers (Corcoran, 1995; Corcoran, et ah, 2003; Caret, et ah, 1999; 
Learning First Alliance, 2003). 



Significantly more Reading First teachers attended professional development workshops related 
to reading than did non-RF Title I teachers (94 percent vs. 81 percent). Further, RF teachers, on 
average, reported having spent significantly more time attending professional development 
activities — conferences, workshops, college courses — in the past year than did teachers in non- 
RF Title I schools (40 hours vs. 24 hours). 

Based on teacher reports, professional development activities attended by RF teachers, as 
compared with non-RF Title I teachers, were significantly more likely to: 

• Offer incentives for participation, such as stipends (40 percent vs. 20 percent), release 
time (43 percent vs. 33 percent), or graduate credits (25 percent vs. 14 percent). 

• Be conducted by well-established and experienced trainers (75 percent vs. 65 
percent). 

• Require teachers to attend (74 percent vs. 57 percent). 

• Use a team-based approach (67 percent vs. 53 percent). 

RF principals, reading coaches and teachers reported having participated in significantly more 
professional development activities to improve their knowledge of the five dimensions of reading 
instruction than did Title I teachers (Exhibit E-6). Eor example, significantly more RE than 
Title I teachers received professional development in phonemic awareness (85 percent vs. 62 
percent). RE teachers rated themselves (on a five-point scale) as significantly better prepared to 
teach the following skills than did Title I teachers: phonemic awareness, 4.13 vs. 3.66; decoding, 
3.86 vs. 3.35; vocabulary, 3.79 vs. 3.40; comprehension, 3.80 vs. 3.54; and fluency, 3.83 vs. 

3.40. 
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Exhibit E-6 



The Five Dimensions of Reading instruction: Reading First and Titie i Teacher Participation in 
and Seif-ratings of Preparedness, 2004-05 Schooi Year 



100 80 60 40 20 

Percent 



1 



□ Reading First 
■ Titie i 




Seif-ratings of preparedness to 
teach the five dimensions of 
reading 



Source: Teacher Survey, Questions D4 and D6 

Exhibit reads: 85 percent of RE teachers participated in professional development on phonemic awareness, compared to 62 
percent of teachers in Title I schools. This difference is statistically significant (p < .05). 

Note: Kindergarten teachers where not asked about fluency. 

The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between teachers in RE and Title I 
schools. 



RF teachers were significantly more likely than teachers in Title I schools to report having 
received professional development assistance on administering and using assessments (85 
percent vs. 67 percent), interpreting assessment data (90 percent vs. 68 percent), and using 
diagnostic tests to guide instruction (66 percent vs. 49 percent). 

Reading coaches and principals also received professional development to support them in their 
roles. RF reading coaches were significantly more likely than their non-RF Title I counterparts 
to report having received professional development assistance on how to help teachers make 
reading instruction systematic and explicit (78 percent vs. 52 percent), and on the essential 
components of SBRR instruction (90 percent vs. 60 percent). Both of these topics are central to 
the Reading First program. Similarly, principals in RF schools were significantly more likely to 
report having received professional development in all five dimensions of reading than were 
principals in Title I schools (p < .05 for all five dimensions). 

Conclusions 

These findings provide some preliminary evidence to suggest that Reading First is being 
implemented in schools and classrooms as intended by the legislation. For the most part, funds 
are awarded to appropriate districts and schools. States are providing appropriate supports. 
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particularly in terms of professional development related to reading, and in the selection and use 
of assessments to inform instruction. Reading First schools appear to have established 
instructional environments to support SBRR-based reading instruction. In K-3 classrooms, the 
reading programs implemented by teachers in Reading First and non-RF Title I schools appear to 
be different in a variety of ways, including instructional time, resources, instructional planning 
and collaboration, use of assessments, and foeus on the five dimensions of reading instruetion. 
Taken together, these findings provide some initial evidence to suggest that Reading First 
schools are carrying out the objectives of the Reading First legislation. Future analyses, after the 
second round of survey data collection in 2007, will examine how implementation of these 
elements changes over time and how student achievement patterns in RF schools may differ from 
those in non-RF Title I schools. 
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Chapter 1 : Introduction 

In October 2003, the Department contraeted with Abt Associates to design and conduct the 
Reading First Implementation Evaluation to address the following questions: 

1. How is the Reading First program implemented in districts and schools? 

2. How does reading instruction differ in Reading First sehools and non-grantee Title I 
schools? 

3. How does reading instruction differ in RF sehools and non-grantee Title I schools as 
RF schools’ implementation efforts mature over time? 

4. Does student achievement improve in schools with Reading First funds? 

5. Is there any relationship between how schools implement Reading First and ehanges 
in reading achievement? 

The present report foeuses on Questions 1-2. Questions 3-5 will be diseussed in the final report. 

Background 

The ability to read and comprehend text well is at the heart of educational attainment and, as 
such, is central to all ehildren’s elementary school success. Unfortunately, success in elementary 
sehool (and beyond) disproportionately eludes many minority and economically disadvantaged 
children. Farge numbers of minority children, often in high-poverty schools, are not developing 
the reading skills needed for success in school. Results from the 2005 National Assessment of 
Educational Progress (NAEP) Reading Assessment indicate that 54 pereent of fourth-grade 
students eligible for free or reduced-price lunches read at a Below Basic level compared to only 
23 pereent of fourth-graders not eligible for free or reduced-price lunehes who perform at that 

o 

level. Although these results are disappointing, these findings are an improvement over the 2000 
NAEP results where 62 percent of students eligible free or reduced priee lunehes seored at the 
Below Basic level (U.S. Department of Education, 2005). 

Scientifically Based Reading Research 

The fact that substantial numbers of our nation’s primary grade students are not developing 
adequate reading skills oeeurs at a time when we have made eonsiderable progress in 
understanding how to teaeh reading effectively in the early grades, particularly to children who 
are struggling academically. The National Researeh Council’s 1998 report. Preventing Reading 
Dijficulties in Young Children, noted that, “the majority of the reading problems faeed by today’s 
adolescents and adults could have been avoided or resolved in the early years of childhood.” The 

* Performance at the Below Basic level means that fourth-grade students are not performing at the Basic level. 
“Fourth-grade students performing at the Basic level should demonstrate an understanding of the overall 
meaning of what they read. When reading text appropriate for fourth-graders, they should be able to make 
relatively obvious connections between the text and their own experiences and extend the ideas in the text by 
making simple inferences.” (National Center for Education Statistics, retrieved May 10, 2006, from 
http://nces.ed.gOv/nationsreportcard/reading/achieveall.asp#grade4). 
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report’ s summary of research on the development of early reading skills concluded that many 
elements of effective reading instruction are already known, and that the provision of “excellent 
instruction is the best intervention for children who demonstrate problems learning to read.” 

Building on the council’s report, the National Reading Panel^ reviewed the scientific research in 
key areas of reading development, focusing on skills critical to the acquisition of beginning 
reading skills. The report found strong evidence that direct, explicit instruction is helpful to 
primary grade children in the development of their reading skills, particularly in the areas of 
phonemic awareness, phonics, vocabulary, comprehension, and fluency. 

The Reading First Legisiation 

The No Child Left Behind Act of 2001 (PL 107-110) established the Reading First Program 
(Title I, Part B, Subpart 1) to address the fact that large numbers of our nation’s students do not 
develop the reading skills necessary to be successful in school. Reading First is a major federal 
initiative that builds on years of scientific research in reading to ensure that all children can read 
at or above grade level by the end of third grade. The legislation requires the U.S. Department of 
Education to contract with an outside entity to conduct an evaluation of, among other things, the 
Reading First (RF) program’s implementation (Section 1205). 

Reading First is predicated on research findings that high-quality reading instruction in the 
primary grades significantly reduces the number of students who experience difficulties in later 
years. The program’s overarching goal is to improve the quality of reading instruction and 
thereby improve the reading skills and achievement of children in the primary grades. The RF 
program provides substantial resources at both the state and local levels: 1) to ensure that 
research-based reading programs and materials are used to teach students in kindergarten through 
third grade; 2) to increase access to and quality of professional development of all teachers who 
teach K-3 students, including special education teachers, to ensure that they have the skills 
necessary to teach these reading programs effectively; and 3) to help prepare classroom teachers 
to screen, identify, and overcome barriers to students’ ability to read on grade level by the end of 
third grade. More specifically, the programs and the professional development provided to 
school staff must use reading instructional methods and materials that incorporate the five 
essential elements of effective primary-grade reading instruction, as specified in the legislation: 

1) phonemic awareness; 2) decoding; 3) vocabulary development; 4) reading fluency, including 
oral reading skills; and 5) reading comprehension strategies. 

In April 2002, the U.S. Department of Education invited state education agencies to apply for 
Reading Eirst grants. State applications submitted to the U.S. Department of Education went 



^ The National Reading Panel (NRP) was formed under the joint auspices of the National Institute of Child 
Health and Human Development and the U.S Department of Education to “assess the status of research-based 
knowledge, including the effectiveness of various approaches to teaching children to read” {Report of the 
National Reading Panel: Teaching Children to Read, 2000, page 1-1.) The findings from the NRP also were 
instrumental in the development of the Reading First program (part of the No Child Left Behind Act), the current 
administration’s comprehensive effort to improve early reading instruction and student reading achievement. 
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through an expert review proeess that resulted in six-year awards dependent on demonstration of 
progress and congressional appropriations. States, in turn, awarded subgrants to local school 
districts based on a competitive process. All^° states and jurisdictions have been awarded 
Reading First grants. To date (April 2006), states have awarded subgrants to approximately 
1,550 local school districts and 5,200 schools nationwide. Because grants to states were 
awarded over an extended time period and states differed in the amount of time allotted to 
subgrant process, districts and schools are at various stages of implementation of their Reading 
First programs. 

The Implementation Evaluation is one of five complementary studies designed to gather 
information about Reading First. In addition to the Implementation Evaluation, the U.S. 
Department of Education’s Policy and Program Studies Service (PPSS) conducted the Analyses 
of State Reading Standards and Assessments that evaluated the alignment of state reading 
content standards for students in grades K-3 with the five key elements of reading instruction by 
analyzing: 1) the reading content standards of a random sample of 20 states; and 2) the role of 
state assessments in measuring Reading Eirst outcomes as presented in state Reading Eirst 
applications. PPSS is also conducting the study of Reading Eirst and Special Education 
Participation Rates, which will use an interrupted time-series design to: 1) compare rates of 
learning disabilities in Reading Eirst schools with a comparison group of schools; 2) investigate 
changes in these rates in RE schools before and after grants were awarded; and 3) examine the 
relationship between reading achievement and rates of learning disabilities. 

The U.S. Department of Education’s Institute of Education Sciences is conducting two studies 
related to Reading Eirst. The Reading Eirst Impact Study is a five-year rigorous evaluation 
designed to measure the impact of Reading Eirst on classroom reading instruction and students’ 
reading achievement. The study is being conducted in more than 250 elementary schools in 18 
sites and 13 states and will collect information on students and classrooms in grades 1-3 over the 
course of three years. The Study of Teacher Preparation in Early Reading Instruction will 
conduct a survey of preservice teachers at 100 schools of education to answer the question: To 
what extent does the content of teacher preparation programs focus on the essential components 
of early reading instruction? In addition, the study will administer an assessment to a random 
sample of 2,000 graduating preservice elementary teachers to answer the question: To what 
extent are graduating preservice teachers knowledgeable about the essential components of early 
reading instruction? 

The Study Design 

The evaluation included the following components: 1) surveys of teachers, principals and 
reading coaches in nationally representative samples of Reading Eirst schools and non-RE Title I 
(non-Reading Eirst) schools; 2) interviews with Reading Eirst state coordinators; 3) the Reading 

*** Guam and Northern Mariana Islands received grants through consolidated grants to the insular areas. 

'* The figures are based on current information listed on ED’s Reading First Awards Database (Retrieved April 1, 
2006, from www.sedl.org/readingfirst/reports-awards.html). 
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First Awards Database that lists all RF districts and schools as well as school and district poverty 
rates and proportions of K-3 students reading below grade level; and 4) ED’s School-Level State 
Assessment Score Database (SLAD) that provides measures of poverty for all school districts 
nationwide. Exhibit 1.1 summarizes the primary data collection activities. The sample design 
calls for two nationally representative samples of Reading Eirst schools — 550 newly funded 
schools that are in their first year of implementation, and 550 mature schools that have been 
implementing RE activities for one year or more — along with 550 Title I (non-Reading Eirst) 
schools.*^ The principal, reading coach (if applicable), and one teacher (randomly selected) from 
each of the four target grades of Reading Eirst (K-3) were sent surveys to complete. Also, the 
study design calls for two waves of data collection in the 2004-05 and 2006-07 school years. 
The present report describes findings based on the data collected in spring 2005. Below we 
describe the survey and state coordinator data collections, including brief descriptions of the 
measures, and include discussions of the sampling strategy and data collection along with the 
corresponding response rates. 



For the purpose of recruiting these two types of RF schools, schools were designated as new or mature based on 
the Reading First program guidelines to states for their annual performance reports; “. . .schools receiving grants 
between July 1 and December 3 1 of any reporting period, the current school year will be considered in the first year 
of implementation.” Therefore, schools awarded subgrants before Dec. 31, 2003, were designated as mature, because 
when data collection occurred in spring 2005, those schools were in at least their second year of implementation. All 
other RF schools were classified as newly funded. 

To produce national estimates from the study sample, all analyses are weighted and computed using STATA, a 
statistical package that generates appropriate standard errors associated with each weighted estimate by taking 
account of the structure of the survey sampling design. 
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Exhibit 1.1 



Data Coiiection Methods, Sampies, and Scheduie 



Data Collection Method 


Number of 
Schools 


Estimated Number of 
Respondents 


Schedule 
2004-05 2006-07 


Principal and Teacher Mail Survey 

Newly funded Reading First schools 


550 


2,200 teachers 






Mature Reading First schools 


550 


550 principals 

up te 550 reading coaches 

2,200 teachers 






Non-RF Title 1 schools 


550 


550 principals 

up te 550 reading coaches 

2,200 teachers 






RF State Coordinator Telephone 




550 principals 

up te 550 reading coaches 

53 state ccordinators 






Interview 











Exhibit reads: 550 newly funded RF schools were selected for inclusion in the study sample. The expected respondents 
include 2,200 teachers, 550 principals, and up to 550 reading coaches. There are two waves of data collection, one in 
spring 2005 and one in spring 2007. 

Note: Four teachers per school were sampled by randomly selecting one teacher from each of grades K-3. One principal 
and one reading coach (if applicable) per school were surveyed. RF state coordinator interviews were conducted with 
respondents in all 50 states, the District of Columbia, American Samoa, and schools run by the Bureau of Indian Affairs. 



Surveys 

Measures 

The primary data collection instruments for the Reading First Implementation Evaluation were 
surveys designed for principals, teachers in grades K-3, and reading coaches. The Principal 
Survey asked for background information on teacher and student characteristics, resources and 
support for reading instruction, reading intervention services, professional development provided 
to teachers and administrators, and principals’ knowledge of research-based approaches to 
reading. The Reading Coach Survey was targeted to school-level individuals whose primary role 
is to assist classroom teachers in delivering effective reading instruction. This survey included 
items on the coaches’ background and experience, core and supplemental reading materials, 
professional development offered to K-3 teachers, specific coaching activities, characteristics of 
reading instruction in the school, changes that have taken place in reading instruction, and areas 
needing improvement. The Teacher Survey addressed teachers’ background and experience, 
student characteristics, reading instruction (e.g., materials, content, time allocation), use of 
assessment, interventions for struggling readers, participation in reading professional 
development, and collaboration and support from other teachers and staff. The surveys also 
included a subset of questions tailored to specific grade levels, the answers to which allowed us 
to describe grade-specific instructional emphases, reflecting RF’s research-based teaching of 
reading. 

Data Collection 

We recruited 1,649 study schools (1,098 Reading First schools and 551 non-RF Title I schools) 
in early 2005. Survey packets (including principal, teacher, and reading coach surveys) were 
mailed to each school. All but six RF schools and 10 Title I schools (1,092 and 541 schools) 
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returned at least one survey/"^ Most schools (88 percent) returned completed surveys for all 
selected respondents in their schools; another 8 percent of schools completed all but one of the 
surveys and only 17 schools, or 1 percent, returned none of the surveys. Response rates were 
slightly higher for Reading First schools compared with Title I schools; 96 percent of Reading 
First schools were either complete or missing only one survey, compared with 93 percent for 
Title I schools. Reading First schools are required, according to the conditions of their awards, to 
participate in a national evaluation, whereas no corresponding requirement exists for Title I 
schools. The response rate across all types of respondents and all schools was 96 percent. Of 
9,460 potential respondents, 9,076 individuals returned completed surveys. For teachers in 
Reading First schools, response rates were approximately 96 percent across the four grade levels, 
compared with 94 percent in Title I schools. All response rates are presented in Appendix A, 
Exhibit A.4.^^ 

Reading First State Coordinator Interviews 

To learn about state-level administration of Reading First, we conducted a semi-structured 
telephone interview protocol for RF state directors. The interview protocols were developed in 
collaboration with the U.S. Department of Education (ED) and were aligned with topics that 
informed states’ development of state Reading Errst plans, including state context for reading 
education, professional development, reading and assessment plans used state- wide and 
differences across districts, the state’s subgrant process, technical assistance, and state 
management and evaluation of the program. Interviews with state Reading Eirst coordinators in 
all 50 states, as well as American Samoa, the schools of the Bureau of Indian Affairs, and the 
District of Columbia, were conducted in April and May of 2005. 

Limitations 

There are several limitations to the findings presented in this report. Eirst, the data sources 
(primarily surveys) represent respondents’ self-reported subjective perceptions and judgments 
about the implementation of their reading programs. The second limitation is based on the 
Reading Eirst legislation which requires states to provide professional development in 
scientifically based reading instruction to all K-3 teachers, not only the teachers in schools that 
receive RE funding; this may reduce the potential to find large differences between RE and non- 
RE Title I schools. 

Third, states often require RE schools to have a designated reading coach, whereas the reading 
coaches in Title I schools are often classroom teachers or reading specialists with additional 
responsibilities. We addressed this shortcoming by limiting the Title I group of reading coaches 
to those respondents who reported that they routinely engage in the activities used to define the 

In addition to the 10 Title I schools that did not complete any surveys, 67 Title I schools refused to participate in 
the study. 

To generate national estimates from the RF and Title I respondent samples, we constructed and applied sets of 
sampling weights at two levels, school and teacher. A discussion of the rationale and method used to construct 
the sampling weights is presented in Appendix A. 
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responsibilities of a reading coach. Nevertheless, any observed differences between RF and 
reading coaches in Title I schools should be interpreted with somewhat more caution than 
observed differences between other respondent categories. Fourth, this report presents findings 
on a large number of comparisons, and the study draws from a large sample; as a result, one 
could expect that there would then be many statistically significant comparisons. We note in the 
text and accompanying tables when specific comparisons are statistically significant, and we also 
note when these comparisons represent statistical differences that are modest, and may not 
substantively meaningful. Finally, this evaluation uses a quasi-experimental design, and as such, 
one cannot attribute observed differences to the Reading First program. 

The Presentation of Findings 

The report is organized into eight chapters, reflecting not only the broad evaluation questions 
(Questions 1 and 2), but the sub-questions related to different facets of implementation of the 
Reading First program as well. Chapter 1 focuses on study design, sampling, and measures. 

The rest of the interim report is organized as follows: 

Chapter 2: Composition of the study sample; 

Chapter 3: Reading instruction; 

Chapter 4: Interventions for struggling readers; 

Chapter 5: Assessment; 

Chapter 6: Oversight and classroom support activities; 

Chapter 7: Professional development; 

Chapter 8: Conclusions. 

The report presents two kinds of findings: descriptions of RF program implementation, and 
comparisons of reading programs in both RF and non-RF Title I schools. The descriptive 
findings use information collected from respondents in the complete sample of RF schools, both 
newly funded and mature, to characterize the Reading First program (Question 1). The 
comparisons are based on the observed differences between the sample of mature RF schools and 
Title I schools (Question 2). 

To describe the implementation of reading programs in RF schools more broadly, we used 
survey data to generate descriptive statistics, such as means, proportions, and frequency 
distributions that are weighted to produce national, population-level estimates.''’ Our study 
sample and design allows us to generate separate findings for each grade, however, when results 
do not vary across grades, we aggregate teacher-level findings across the four grades and, to 
simplify the presentation, we display the range of grade-level means. Specific grade-level 
estimates are, however, presented in Appendix C. 



All weighted analyses are conducted using STATA, a statistical package that generates appropriate standard 
errors associated with each weighted estimate by taking account of the structure of the survey sampling design. 
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For the purpose of recruiting the two types of RF schools, schools were designated as new or 
mature based on the Reading First program guidelines to states for their annual performance 
reports.'^ 



. schools receiving grants between July 1 and December 3 1 of any reporting period, the current school year 
will be considered in the first year of implementation.” Therefore, schools awarded subgrants before Dec. 31, 
2003, were designated as mature, because when data collection occurred in spring 2005, those schools were in 
at least their second year of implementation. All other RF schools were classified as newly funded. 
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Chapter 2: Composition of the Study Sample 



This chapter examines the characteristics of the sampled schools, both those that have received 
RF funding and the non-RF Title I schools. In this section, we present background information 
on school enrollment, staffing, student populations and external resources targeted to reading 
schools’ reading programs. 

Characteristics of Reading First and Title I Schools 

This section describes RF and Title I schools in terms of school characteristics (e.g., size, 
attendance and mobility rates), staff experience, students, resources targeted to reading 
programs, and accountability requirements of A^o Child Left Behind (NCLB). Sampling weights 
have been applied to the survey data to provide estimates that represent the populations of RF 
and Title I schools. 

School Characteristics 

Exhibit 2.1 presents the distribution of schools based on school size; the distributions are similar 
for the populations of RF and Title I schools. There is a significantly greater proportion of very 
large RF schools than Title I schools (17 percent vs. 11 percent). This difference is also reflected 
in the mean enrollment, which, on average, is significantly higher in RF schools. 



Exhibit 2.1 



Schooi Enroiiment and Urbanicity in Reading First and Titie i Schoois, 2004-05 Schooi 
Year 





Reading First Schools 


Title 1 Schools 


School Size 


Mean enrollment 


513* 


465 




Percent 


Percent 


Very small (1-99) 


3% 


3% 


Small (100-249) 


11 


14* 


Medium (250-499) 


40 


46* 


Large (500-749) 


29 


26 


Very large (750+) 


17* 


11 


Urbanicity 


Urban 


46%* 


39% 


Suburban 


34 


33 


Rural 


20 


28* 



Source: Principal Survey, Question A3a. 

Weight: Principal. 

Weighted respondents: 3,878 RF principals, 14,625 Title I principals. 

Nonresponse rates across survey items: < 1 percent. 

Exhibit reads: About 3 percent of RF and non-RF Title I schools are very small, with enrollments of less than 100. 
Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 
Title I schools. 
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Enrollment stability is similar for RF and Title I schools, with approximately one-quarter of the 
schools in each group experiencing an increase in enrollment and another quarter of the schools 
experiencing a decrease in enrollment over the last five years (Exhibit 2.2). Reading First and 
Title I schools also have similar attendance and mobility rates. Average attendance across both 
types of schools is about 94 percent. Mobility rates average 18 percent for RF schools and 16 
percent for Title I schools. This difference, while statistically significant, is not substantively 
large or meaningful. 



Exhibit 2.2 

Mobiiity Rates, Attendance Rates and Changes in Enroiiment in Reading First and 
Titie i Schoois, 2004-05 Schooi Year 




Reading First 


Titie 1 




Schoois 


Schools 




Percent 


Percent 


Change in Enroiiment in Last Five Years 

Decreased 


47% 


44% 


Remained stable 


25 


27 


Increased 


27 


28 


School is new 


2 


1 



Mobility Rate 18%* 16% 

Attendance Rate 94 95 

Source: Principal Survey, Questions A3c, A3d, and A5. 

Weight: Principal. 

Weighted respondents: Mobility and attendance rates — 3,186 RF principals, 12,305 Title I principals; .change 
in enrollment: 3,790 RF principals, 14,032 Title I principals. 

Nonresponse rates across survey items: 4.2 to 9.5 percent. 

Exhibit reads: Attendance rates in both Reading First and Title I schools during the 2004-05 school year 
were, on average, 94 and 95 percent, respectively. This difference is not statistically significant. 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF 
and Title I schools. 



School Staff 

Overall, staff reports of their years of experience are similar in RF and Title I schools. 
There are, however, some small differences. Principals in Title I schools have significantly 
more experience as principal than do their counterparts in RF schools (8.5 years vs. 7.7 years). 
Similarly, they have been in their current schools slightly longer than principals in RF schools 
(5.3 years vs. 4.8 years). While these differences in experience are statistically significant, they 
are not substantively large (Exhibit 2.3). Although, on average, the principals appear to be 
experienced, about half of the principals have been in their schools for three years or less (5 1 
percent of the RF principals and 46 percent of the Title I principals). Further, 23 percent of RF 
principals and 18 percent of Title I principals reported that 2004-05 represented their first year in 
that school as principal (Exhibit 2.3). Frequent changes in key staff, such as the principal, may 
well have an effect on how well new programs function in the school. 
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Exhibit 2.3 

Years of Experience for Staff in Reading 
Year 


First and Titie i Schoois, 2004- 


-05 School 




Reading First 


Title 1 




Schools 


Schools 




Mean 


Mean 


Principals 


Years experience as principal 


7.7 


8.5* 


Years in this scheol 


4.8 


5.3* 


Teachers 


Years experience 


12.8 


13.2 


Years in this scheol 


8.0 


8.5 


Reading Coaches' 




Percent 


Percent 


Schools with Reading Coaches 


98%* 


60% 




Mean 


Mean 


Years experience 


18.0 


18.5 


Years in this school 


7.5 


9.0* 


Years as reading coach In this school 


1.8 


3.3* 



Source: Principal, and Teacher Surveys, Question Al; Reading Coach Survey, Question A3. 

Weight: Principal, Teacher, and Reading coach. 

Weighted respondents: 3,866 RF principals, 14,485 Title I principals; 64,545 RF teachers, 201,431 Title I 
teachers; 3,733 RF reading coaches, 7,613 reading coaches in Title I schools. 

Nonresponse rates across survey items and respondents: 0.3 to 5.2 percent. 

Exhibit reads: Principals in Reading First schools have, on average, 7.7 years experience in that position, 
compared with 8.5 years for principals in Title I schools. This difference is statistically significant (p = < .05). 
Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF 
and Title I respondents. 

' 34 reading coaches in Title I schools were excluded from this analysis because based on their survey 
responses, they do not appear to meet the definition of “reading coach” used in this evaluation. 



Teachers in RF and Title I schools are equally experienced, with about 13 years of teaching 
experience, on average. About one-third of teachers reported that they are relatively new to 
the current schools — 14 percent of teachers for one year or less; 18 to 20 percent of 
teachers have taught in this school for between one and three years (See Appendix C.2.a for 
details). 

Reading coaches in RF and Title I schools are even more experienced than teachers in these 
schools; on average, they have 18 years of combined teaching or coaching experience 
(Exhibit 2.3). Further, 75 percent of reading coaches, whether in Reading First or Title I 
schools, have ten or more years of experience. Reading coaches in Title I schools reported that 
they have been working in their current schools for more than three years; this compares with 
less than two years for RF coaches, reflecting a distinction between the RF and Title I schools. 
Most states require RF schools to employ reading coaches to support teachers’ reading 
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instruction; this is not the case for non-RF Title I schools, which may or may not have reading 
coaches. 

Preservice Teacher Training in the Five Dimensions of Reading 

A cornerstone of the Reading First program is that teachers should be knowledgeable about and 
well prepared to teach the five essential components of reading instruction — phonemic 
awareness, decoding, vocabulary, comprehension, and reading fluency. Teachers were asked to 
rate the extent to which their preservice training prepared them to teach the five dimensions of 
reading using a scale of 1 (not at all prepared) to 5 (extremely well-prepared). Generally, across 
the five dimensions, RF teachers rated themselves in the middle of the range — suggesting 
that their preservice training left them somewhat prepared to teach these skills. On all 
dimensions. Title I teachers rated themselves significantly higher than did the RF teachers; 
these differences, while statistically significant, are substantively small. (Exhibit 2.4). 



Exhibit 2.4 



Teachers’ Self-Ratings on Their Preservice Training to Teach Five Dimensions of 
Reading: Teachers in Reading First and Title 1 Schools, 2004-05 School Year 




RF Teachers 


Titie 1 Teachers 


Dimension 


Mean Self-Rating 


Mean Self-Rating 


Phonemic awareness 


2.99 


3.15* 


Decoding 


3.10 


3.19* 


Vocabulary 


3.25 


3.45* 


Comprehension 


3.33 


3.47* 


Fluency 


2.89 


3.09* 



Source: Teacher Survey, Question A5. 

Weight: Teacher. 

Weighted respondents: 64,397 RF teachers, 201,292 Title I teachers. 

Nonresponse rate across survey items: < 1 percent. 

Exhibit reads: Title I teachers rated themselves as better prepared based on their preservice training to 
teach phonemic awareness than did RF teachers (3.15 vs. 2.99, p is < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between 
RF and Title I teachers. 

Ratings are on a scale of 1 to 5, where 1= not at all prepared and 5 = extremely well prepared. 



Student Population 

Principals reported that speeial edueation serviees are provided to roughly the same proportion of 
students in RF and Title I sehools. In both groups of sehools, this proportion inereases from 
about 5 pereent in kindergarten to almost 10 pereent for third grade students (Exhibit 2.5). 
Prineipals also reported that significantly more students in RF schools are also more likely to 
receive English as a Seeond Eanguage (ESE) edueation serviees — about 20 percent of K-3 
students, eompared with about 10 pereent for students in Title I sehools. 
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Exhibit 2.5 



Student Characteristics in Reading First and Titie i Schoois, 2004-05 Schooi Year 





Reading First 
Schools 


Title 1 
Schools 




Mean Percent 


Mean Percent 


Receive Special Education Services 


Kindergarten 


6% 


6% 


1st grade 


7 


7 


2nd grade 


8 


9 


3rd grade 


10 


10 


Receive ESL Instruction 


Kindergarten 


22* 


12 


1st grade 


21* 


12 


2nd grade 


21* 


11 


3rd grade 


20* 


10 


Instruction in language other than English 


Kindergarten 


7* 


6 


1st grade 


7* 


5 


2nd grade 


6 


5 


3rd grade 


5* 


4 


Source: Principal Survey, Question A6. 
Weight: Principal. 


Weighted Respondents: 3,551 RF principals, 13,499 Title I principals. 
Nonresponse rates across survey items and grades: 7.8 to 13.5 percent. 

Exhibit reads: In RF schools and Title I schools, 6 percent of kindergarten students 


receive special 


education services. 


Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF 


and Title I schools. 



External Resources to Support Schools’ Reading Programs 

Reading First is the signature reading program of the No Child Left Behind (NCLB). As such, it 
represents a substantial investment in improving the reading achievement of the nation’s 
students. However, states, school districts, and schools receive support for their reading 
programs from other sources. Interviews with state Reading First coordinators, for example, 
found that 30 states had separate reading initiatives based on scientifically based reading 
research (SBRR), and 36 states had major statewide initiatives focused on professional 
development in reading instruction for teachers or reading specialists and administrators, 
among other initiatives. Below, we summarize survey results about the array of funding and 
external support for reading for Reading First schools. 

Size of Reading First Grant 

For the total population of Reading First schools, principals reported that the median 
annual RF award was $138,000 for school year 2004-05 (Exhibit 2.6). Typically, schools 
receive funds for three years or more. The funds ranged in size from a low of $2,000 to a 

** Other initiatives include revised reading or language arts standards and accountability or assessment initiatives 
focused on reading proficiency (29 states), early child education and school readiness initiatives (18 states), and 
family literacy programs such as Even Start. 
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high of $854,000. About 12 percent of the awards were less than $50,000; less than 1 percent 
were more than $500,000. The great majority of the RF awards to schools (81 percent) were 
between $50,000 and $299,999. We expected that larger schools, on average, would receive 
larger Reading First awards, and this trend seems to hold; however, some smaller schools 
received larger awards as well. 



Exhibit 2.6 



Distribution of Reading First Funds to Schoois by Average Enroiiment, for Reading First 
Schoois, 2004-05 Schooi Year 

Median RF award to schools Reading First Schools 



Size of annual RF award 


Percent of 
Schools 


Average 

School Enrollment 


... $2,000— $99,999 


29% 


470 


...$100,000-$1 99,999 


46 


490 


...$200,000-$299,999 


18 


511 


...$300,000-$399,999 


4 


532 


...$400,000-$499,999 


2 


715 


...$500,000 and over 


1 


737 



Source: Principal Survey, Question B6. 

Weight: Principal. 

Weighted respondents: 2,851 RF principals. 

Nonresponse rate: 27.1 percent. 

Exhibit reads: 29 percent of Reading First schools received funds of between $2,000 and $99,999. The average school 
enrollment for schools receiving between $2,000 and $99,999 in Reading First funds was 470 students. 



Other Sources of Financial Assistance for K-3 Reading Programs 

Principals of the total population of Reading First schools reported that they received funding 
from an average of 5. 1 sources, compared with 4.5 sources reported by principals of Title I 
schools (Appendix C, Exhibit 2.C.6). Significantly more Title I schools than Reading First 
schools reported receiving funding from Title I, district general funds, state textbook funds, 
professional development funds, and private grants. 

Nonfinancial Assistance with K-3 Reading Programs 

Beyond financial support, RF principals were significantly more likely to report receiving 
substantially more external, nonfinancial assistance than Title I principals, in a variety of 
areas including selecting instructional programs (76 percent vs. 56 percent), diagnosing 
needs of struggling readers (70 percent vs. 50 percent), conducting demonstration lessons 
(71 percent vs. 48 percent), and reviewing the effectiveness of reading programs (71 
percent vs. 47 percent) (See Exhibit 2.7). 
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Exhibit 2.7 



Nonfinanciai Externai Assistance for K-3 Reading Program Activities in Reading First and 
Non-RF Titie i Schoois, 2004-05 Schooi Year 





Reading First 


Title 1 




Schools 


Schools 


Type of Assistance 


Percent 


Percent 


Planning professional development 


84%* 


73% 


Interpreting assessment results 


84* 


70 


Conducting classroom observation 


80* 


49 


Providing technical assistance in implementing cere reading programs 


81* 


50 


Selecting prefessional develepment providers 


77* 


57 


Selecting assessment instruments 


76* 


56 


Selecting instructional programs/materials 


76* 


56 


Reviewing reading program effectiveness 


71* 


47 


Conducting demonstration lessons 


71* 


48 


Diagnosing needs of struggling readers 


70* 


50 


Setting up intervention programs for struggling readers 


70* 


51 


Providing technical assistance for using supplementary reading materials 


69* 


48 


Conducting needs assessment for professional development 


57* 


43 


Leading teacher study groups 


49* 


33 


Recruiting staff with reading expertise 


40* 


25 



Source: Principal Survey, Question B7. 

Weight: Principal. 

Weighted respondents: 3,850 RF principals, 14,481 Title I principals. 

Nonresponse rates across survey items: 1.7 to 4.4 percent. 

Exhibit reads: 82 percent of principals in RF schools reported receiving external assistance in interpreting assessment results, 
compared to 70 percent of principals in Title I schools. This difference is statistically significant (p-value < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
schools. 



NCLB Accountability 

All schools, including Reading First schools, must also continue to meet the aeeountability 
requirements of the NCLB legislation. NCLB mandates that states develop and implement 
systems of accountability to ensure that districts and schools make adequate yearly progress as 
measured by the aeademie aehievement of its students (Part A, See. 1111, (b), (2)). States 
develop their own definitions of Adequate Yearly Progress (AYP) for their distriets and sehools 
by specifying the minimum levels of improvement in student performanee that must be attained. 
The legislation also ineludes sanetions for sehools that do not make AYP. Sehools that do not 
make AYP for two conseeutive years are designated as sehools in need of improvement and are 
required to develop a plan to remedy the situation; they are in year 1 of sehool improvement. If, 
at the end of year 1 a sehool fails to make AYP, the sehool distriet is required to provide 
teehnical assistanee and supplemental educational services to the eligible students in those 
sehools. Then, if those sehools fail to make AYP at the end of year 2 of improvement, the distriet 
must implement a series of ‘eorrective aetions’ (e.g., replaeing staff, implementing new 
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curricula, extending the school day, restructuring the organization of the school). If a school does 
not make AYP after five or more years, the school must implement a restructuring plan. 



Of the 4,764 schools identified as RF schools during the 2004-05 school year, currently about 
1 ,096 (or 23 percent) have been designated as schools in need of improvement, exactly the 
schools RF is supposed to serve (Exhibit 2.8). About 70 percent of the RF schools in need of 
improvement are in their first or second year of school improvement; about 19 percent are 
schools in which districts are required to implement corrective action to improve the school. In 
about 1 1 percent of these schools, districts are required to restructure schools in which AYP has 
not been accomplished for five years. 



Exhibit 2.8 

Status of Reading First Schoois Designated as in Need of improvement, 2004-05 Schooi 
Year 





Number of Schools 


Percent 


School improvement — Year 1 


418 


41% 


School improvement — Year 2 


294 


29 


Corrective action 


189 


19 


Restructuring and planning 


113 


11 


Total 


1,014 


100 



Source: Database provided by the U.S. Department of Education. 

Note: 82 RF schools included in the Schools in Need of Improvement database are missing a specific status 
classification. 



Summary 

The demographic characteristics of Reading First and Title I schools are similar in several areas 
including attendance rates, mobility, and stability of enrollment. In general, the patterns of staff 
experience are similar for the two groups of schools. About half of the principals and one-third 
of the teachers have been in their current schools for three years or less. Reading First schools, 
however, on average have larger enrollments and are more likely to be located in urban areas 
with almost half of RF schools in such a locale, compared with 39 percent of Title I schools. 

The K-3 student populations of RF and Title I schools are similar in terms of the proportion of 
students receiving special education services and instruction in a language other than English. 
However, these schools differ in their proportions of students receiving ESE instruction; about 
one-fifth of K-3 students in RE schools receive such services, compared with about 10 percent in 
Title I schools. 

The median school in the total Reading Eirst population received $138,000 in federal Reading 
Eirst funds for school year 2004-05; about 80 percent of the awards were between $50,000 and 
$299,999 in size. In addition to these funds, most schools in the total population of Reading Eirst 
schools received support for reading programs from Title I (91 percent) and from school district 
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general funds (79 percent). Reading First schools also received many different kinds of 
nonfinancial assistance for their K-3 reading programs in the form of assistance with planning 
professional development, interpreting assessment results, implementing the core reading 
program, and conducting classroom observations. Finally, across a variety of types of 
nonfinancial assistance, Reading First schools were much more likely to receive such assistance 
than were Title I schools. 
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Chapter 3: Reading Instruction 



The purpose of Reading First is to change how reading is taught in K-3 classrooms. These 
changes may include how much time is allocated to reading instruction, the types of materials 
used for reading instruction, the strategies used for helping teachers implement reading 
instruction, and the specific activities teachers use to teach reading. 

This chapter reports findings on the amount of time spent on reading instruction in Reading First 
and Title I schools, changes in materials and the types of materials used for reading instruction, 
and the types of strategies and activities used for reading instruction in Reading First and Title I 
classrooms. 

Instructional Time 

Research has shown that the amount of time that schools spend on reading is a major determinant 
of reading achievement (National Research Council, 1998). The Reading First program guidance 
notes that schools “should also consider the allocation of time, including a protected, 
uninterrupted block of time for reading instruction of more than 90 minutes per day.”*^ This 
section presents results on whether uninterrupted reading blocks are scheduled, the amount of 
time scheduled and actually spent on reading instruction, and changes in the amount of time 
spent on reading instruction compared to the previous school year, as reported by reading 
coaches and classroom teachers. 

Amount of Time Spent on Reading Instruction 

Reading coaches in RF and Title I schools agreed that there is sufficient time during the school 
day allotted for reading instruction. As shown in Exhibit 3. 1, a great majority of reading 
coaches and principals (88-98 percent) reported that their schools had a scheduled reading block 
for grades 1-3. However, mature Reading First schools were significantly more likely than 
Title I schools to have a scheduled reading block (98 percent vs. 92 percent). At these three 
grade levels, Reading First schools, on average, reported having a 15-minute longer 
scheduled reading block than did Title I schools. Very few grade 1-3 classrooms in mature 
Reading First schools (3 percent) reported having a scheduled reading block less than 90 
minutes long, compared with more than 20 percent of the grade 1-3 classrooms in Title I 
schools. 

These differences between mature Reading First and Title I schools were similar for kindergarten 
classrooms. Kindergartens in Reading First schools were also significantly more likely to have a 



U.S. Department of Education, Office of Elementary and Secondary Education, “Guidance for the Reading Eirst 
Program.” Washington, D.C., April 2002, page 6. 

Reading coaches were asked to indicate on a scale of one to five how accurately the statement “Sufficient time 
during the school day is allotted for reading instruction (Question HI)” describes their school. The average 
response was 4.54 for RE schools and 4.46 for Title I schools. This difference is not statistically significant. 
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scheduled reading block than Title I kindergartens (98 percent vs. 88 percent). Although nearly 
12 percent of RF kindergartens had scheduled reading blocks less than 90 minutes long, 
nearly one-third of Title I kindergartens (33 percent) had reading blocks of less than 90 
minutes. 

Teachers in Reading First schools, at all four grade levels, reported that they spent more 
time on reading than did teachers in Title I schools — a difference of about 19 minutes per 
day, or almost 100 minutes per week (99 minutes vs. 80 minutes per day). 



Exhibit 3.1 

Percentage of Mature Reading First Schoois and Titie i Schoois with a Daiiy Scheduied Reading 
Biock, Length of the Reading Biock, and Totai Length of Reading instruction, 2004-05 Schooi 

Year 

Reading First Title I 

Grade Level Schools Schools 

Kindergarten 




Source: Reading Coach Survey, Questions D2 and D3; Principal Survey, Questions D2 and D3. 

Weight: School. 

Weighted respondents: 2,216 RF reading coaches or principals, 14,259 reading coaches or principals in Title 1 schools. 
Nonresponse rates across survey items and respondents: 0 to 5 percent. 

Exhibit reads: 98 percent of RF schools reported having a scheduled reading block at the kindergarten level, compared with 88 
percent of Title I schools. The difference is statistically significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title 1 
schools. 

Total reading instruction includes the scheduled reading block. Data are taken from Reading Coach survey. If the Reading 
Coach survey is missing responses to this question, then data are taken from Principal Survey. 



Perceived Change in Amount of Time Spent on Reading Instruction 

About 60 percent of the K-3 teachers in new Reading First schools reported that they 
increased the amount of time spent on reading in 2004-05 as compared with 2003-04; 
additionally, around 45 percent of the K-3 teachers in mature Reading First schools 
reported increased time spent on reading compared to the prior year (Exhibit 3.2). In 
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comparison, about one-third of K-3 teachers in Title I schools reported increasing the 
amount of time spent on reading instruction in 2004-05. 



Exhibit 3.2 



Percentage of Teachers in Mature and New Reading First Schoois and Titie i Schoois Who 
Changed the Amount of Time Spent Teaching Reading from Previous Year, 2004-05 Schooi 
Year 



Change in Time Spent 


Mature Reading First 
Schools 


New Reading First 
Schools 


Title 1 
Schools 


Teaching Reading 


Percent 


Percent 


Percent 


Kindergarten teachers 








...Increase 


44% 


60%* 


40% 


...Remain the same 


54 


37 


55* 


...Decrease 


2 


3 


5 


1st grade teachers 








...Increase 


47 


59* 


34 


...Remain the same 


52 


36 


61* 


...Decrease 


2 


5 


5 


2nd grade teachers 








...Increase 


47 


62* 


34 


...Remain the same 


51 


35 


61* 


...Decrease 


2 


3 


5 


3rd grade teachers 








...Increase 


42 


61* 


31 


...Remain the same 


55 


35 


63* 


...Decrease 


3 


3 


7 



Source: Teacher Survey, Question C2. 

Weight: Teacher. 

Weighted respondents: 35,215 teachers in mature Reading First schools, 23,908 teachers in new Reading First schools, 
187,373 teachers in Title I schools. 

Nonresponse rate across grades: < 1 percent. 

Exhibit reads: 47 percent of first-grade teachers in mature RF schools increased the amount of time they spent teaching 
reading from the previous year. 59 percent of first-grade teachers in new RF schools increased the amount of time they 
spent teaching reading, compared with 34 percent of first-grade teachers at Title I schools. This difference is statistically 
significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between new RF and 
Title I schools. 



Instructional Materials 

The Reading First legislation provides states and districts with specific guidelines about the 
selection of instructional materials for teaching reading. In their grant applications, states were 
required to demonstrate how they would support districts’ efforts to identify instructional 
materials based on scientifically based reading research. The guidance further specifies that a 
high-quality reading program must have a coherent design and address the five essential 
elements of reading instruction. In this section we describe I) how RF schools selected their 
reading instructional materials and 2) the instructional programs (core, supplementary, and those 
selected for English Language Learners) used most frequently in RL schools. 

The self-report information provided by the survey respondents suggest that RL schools have 
made changes to their reading programs. Reading coaches and school principals indicated 



Chapter 3: Reading Instruction 



37 








whether they had ehanged their reading program materials sinee the beginning of the 2004-05 

sehool year. RF reading coaches were significantly more likely to indicate that K-3 classrooms 

21 

had ample, high quality instructional materials than were coaches in Title I schools. 

As shown in Exhibit 3.3, new RF schools were significantly more likely than their Title I 
counterparts to have adopted a new core reading program at the beginning of the school 
year (39 percent vs. 16 percent). They were also much more likely to have added new 
intervention programs for struggling readers (74 percent vs. 43 percent), new 
supplementary materials (69 percent vs. 58 percent), new materials for English Language 
Learners (43 percent vs. 23 percent), and new reading assessments (71 percent vs. 36 
percent). Data from mature RF schools are not presented, because mature RF schools would 
most likely have adopted a new core reading program at the start of their RF grant and not 
changed their core program as of the beginning of the 2004-05 school year. 



Reading coaches were asked to indicate on a scale of one to five how accurately the statement “K-3 classrooms 
have ample, high quality instructional materials” (Question HI)” describes their school. The average response 
was 4.39 for RF reading coaches and 4.01 for reading coaches in Title I schools. This difference is statistically 
significant (p < .05). 

The figures for mature Reading First schools are as follows: 7 percent reported adopting a new core reading 
program in the 2004-05 school year; 52 percent reported adding a new intervention program for struggling 
readers; 57 percent reported adding new supplementary materials; 30 percent reported adding new materials for 
ELLs; and 26 percent reported adding new reading assessments. 

For this study, a core reading program is defined as one that provides a comprehensive program of instruction 
on a daily basis in all aspects of reading. Supplementary reading materials provide instruction in a targeted area 
of reading to all students. Intervention programs are designed to help struggling readers and are to be used in 
addition to the core reading program. 
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Exhibit 3.3 



Changes to Reading Program Materiais for New Reading First and Titie i Schoois, 2004-05 Schooi Year 




Adopted a new Added a new Added new Added new Adopted new 

core reading intervention supplementary materials for ELLs reading 

program program for materials assessments 

struggling readers 

Source: Reading Coach Survey Question C3; Principal Survey, Question D7. 

Weight: School. 

Weighted respondents: 18,239 reading coaches and principals. 

Nonresponse rates across survey items: 1 to 2 percent. 

Exhibit reads: 39 percent of the new Reading First schools in the sample had adopted a new core reading program at the beginning of 
the school year, as opposed to 16 percent of Title I schools in the sample. This difference is statistically significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I schools. 



Responsibility for Selection of Materials 

Interviews with state Reading First coordinators indicated that virtually all states were involved 
in the selection of core reading programs for Reading First schools, either by developing a list of 
recommended reading programs (37 states), or by referring districts to A Consumer’s Guide for 
Evaluating a Core Program Grades K-3^^ (15 states). However, although most state agencies 
were involved in choosing programs, much flexibility existed for districts and schools. 

Twenty-eight states reported providing assistance to districts or schools in selecting a core 
reading program. Most of these states (23) provided this assistance prior to subgrant award, 
usually by providing lists of approved programs or other guidance (such as directing districts or 
schools to the Consumer’s Guide. 

Principals too were asked about who was involved in selecting instructional materials. As shown 
in Exhibit 3.4, 86 percent of principals reported the selection of core reading programs involved 



Reading First coordinators for the 50 states plus the District of Columbia, American Samoa, and the Bureau of 
Indian Affairs schools were interviewed. 

A Consumer’s Guide for Evaluating a Core Program Grades K-3: A Critical Elements Analysis by Deborah 
Simmons and Edward Kame’enui is a product of the National Center to Improve Tools for Educators and the 
Institute for the Development of Educational Achievement at the College of Education, University of Oregon. 
The Guide lists criteria that should be carefully considered in selecting a scientifically based core reading 
program. 
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the district. Forty-two percent of the RF principals reported that districts selected the core 
reading programs without the involvement of other actors. Thirty percent of the RF principals 
reported that the state was involved in the core reading program selection process, although 
principals may not have been aware of state involvement in providing menus from which to 
select programs. 

Approximately one-third of principals in the total population of Reading First schools reported 
that they or their reading coach had participated in the selection of the core reading program. 
Very few schools (7 percent) reported that they were able to select core programs without any 
state or district input. This suggests that the RF guidelines that encourage state and district 
participation in key instructional decisions are largely being followed. 



Exhibit 3.4 

Responsibiiity for Seiection of Reading Materiais in RF Schoois, 2004-05 Schooi Year 




Selection of intervention Selection of Selection of a specific 

reading program supplemental reading core reading program 
materials program materials 



Source: Principal Survey, Question D4. 

Weight: Principal. 

Weighted respondents: 3,91 1 RF principals. 

Nonresponse rates across survey items: < 1 percent. 

Exhibit reads: 69 percent of RF school principals indicated that their school’s reading coach was involved in the selection 
of intervention reading program materials for their school. 



Principals reported that the districts were also involved in the selection of supplemental materials 
(66 percent), although the district selected supplemental materials without involvement from the 
school or the state for only 15 percent of sehools. A smaller proportion of states helped select 
supplemental curriculum materials (19 percent). Principals more commonly reported that it was 
school personnel, either the reading coaches or principals, who selected supplemental materials 
(61 and 66 percent, respectively). In fact, in nearly one-third of schools (28 percent), school staff 
selected the supplemental curriculum materials without either state or district involvement. The 
selection patterns of interventions for struggling readers mirror those described above. 

The actors involved in the selection of curricular materials differ in mature RF and Title I 
schools. As shown in Exhibit 3.5, principals in mature RF schools were more likely than 
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principals in Title I schools to report state involvement in the selection of core reading programs 
(31 percent vs. 18 percent), supplemental materials (19 percent vs. 6 percent), and intervention 
reading materials (18 percent vs. 5 percent). 



Exhibit 3.5 

Responsibiiity for Seiection of Reading Materiais in Mature Reading First and 




State District Principal Reading coach 



Percent of Selection of Supplementai Reading Program Materiais 

Schools 




Percent of Selection of Intervention Reading Program 

Schoois Materiais for use with Struggling Readers 




Source: Principal Survey, Question D4. 

Weight: Principal. 

Weighted respondents: 2,210 RF principals, 14,684 Title I principals. 

Nonresponse rate across survey items: < 1 percent. 

Exhibit reads: 31 percent of principals in RF schools indicated that the state was involved in the selection of the school’s 
core reading program vs. 18 percent of principals in Title I schools. This difference is statistically significant (p < .05). 
Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 
Title I schools. 
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Principals in mature RF schools were significantly more likely than Title I principals to report 
district involvement in selecting core materials (88 percent vs. 83 percent) and intervention 
materials (64 percent vs. 56 percent). These differences, although statistically significant, are not 
substantively meaningful. There were no significant differences between mature RF and Title I 
schools in district involvement for selection of supplemental materials. This may reflect the fact 
that core materials have not been in place long enough in Reading First schools for districts to 
systematically review what needs to be supplemented. Not surprisingly, reading coaches were 
more likely to be involved in the selection of materials in RF schools than in Title I schools, 
reflecting the fact that most states required their RF schools (but not their Title I schools) to hire 
reading coaches. Finally, principals in RF and Title I schools reported statistically similar 
patterns of involvement in the selection of supplemental materials. (Details are shown in 
Appendix C, Exhibit C.3.5.) 

Core Reading Programs 

One of the cornerstones of the Reading First program is the implementation of a core reading 
program that is supported by SBRR and provides instruction in the five essential elements of 
reading. A core reading program is one that provides a comprehensive daily program of 
instruction in all aspects of reading. 

Each state is responsible for ensuring that the core reading programs the Reading Eirst schools 
use, are aligned with scientifically based reading research. Some states fulfill that responsibility 
by developing a list of core reading programs from which districts or schools make their 
selections. Others allow local selection. Of the 37 state Reading Eirst coordinators who reported 
the core reading programs used by name, almost all (35) reported three or more programs in use 
in their states. Two states limit their allowable core programs to two. The states with the shorter 
lists contain 20 percent of the Reading Eirst schools in the study sample. 

We asked principals and reading coaches in an open-ended question about the core program in 
use in each grade K-3. Both principals and reading coaches were surveyed because not all 
schools have reading coaches. Where available, we used the reading coach response. We 
received responses from 100 percent of the Reading Eirst schools versus 81 percent of the 
sampled Title I schools. Response rates were lower for this question than many others in the 
report because it was open-ended^^ and possibly because some sampled Title I schools^^ or 
grades did not have a core reading program. 



Some respondents omitted the name of the publisher, the name of the program, or the year of the program. 

As is true with all of the survey results, the Title I sample includes only Title I schoolwide project (SWP) 
schools with at least 40 percent of the students eligible for free or reduced price lunches. Therefore this 
information is not generalizable to all Title I schools. 
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Reading First personnel reported using 37 different eore reading programs (Exhibit 3. 6. a). The 
most frequently cited programs in Reading First schools include: Houghton Mifflin 
(unspecified), McGraw-Hill Reading, Nation’s Choice, Open Court, Success for All, and 
Trophies. The findings for the sample of Title I schools were similar. Personnel in the sampled 
Title I schools mentioned 49 different core reading programs (Exhibit 3.6.b). The most 
frequently cited programs in these Title I schools were: McGraw-Hill Reading, Open Court, 
Scott Foresman Reading, Success for All, and Trophies. 



Exhibit 3.6.a 



Core Reading Programs Used by Reading First Schools, 2004- 


-05 School Year 


Reading First Schools 


Percentage of Schools in Sample that 
Use the Program 


Publisher 


Program 


1-5% 


Addison Wesley 


Unspecified 




Harcourt 


Unspecified/Other 
Rigby Reading 
Collections 




Houghton Mifflin 


Lecture 
Reading 2005 
State Specific Edition 




McGraw-Hill 


Unspecified 




Scott Foresman 


State Specific Edition 
Reading 
Unspecified 
Literacy Works 




Sopris 


Read Well 




Voyager 


Universal Literacy 




Houghton Mifflin 


Legacy of Literacy 


6-1 0% 


Houghton Mifflin 


Unspecified 




McGraw-Hill 


Reading 




Success for All 


Success for All 


11-23% 


Harcourt 


Trophies 




Houghton Mifflin 


Nation's Choice 




McGraw-Hill 


Open Court 



Source: Reading Coach Survey, Principal Survey Question Cl. 

Unweighted. 

Respondents: 1,042 RF reading coaches and 50 RF principals. 

Nonresponse rate: 0 percent. 

Exhibit reads: Between land 5 percent of RF schools (reading coaches or principals) reported using a reading program 
published by Addison Wesley for at least one of the grades K-3. 

Notes: Programs with frequencies of less than 1 percent are presented in Appendix C, Exhibit C.3.6.a. 

The columns list the unweighted percent of schools that listed the specified reading program as the core program their school 
uses for at least one of grades K-3. 

Weights were not used because many strata did not have enough respondents to accurately figure the weighted percent. 

In many instances respondents identified a publisher but did not specify a program. These are designated here as 
“unspecified.” Percentages have been rounded to the nearest whole percent. 

A number of publishers have developed programs tailored specifically to the needs of individual states, referred to here as 
“state specific editions.” 



3.9 percent of the Reading Coach respondents named multiple publishers for an individual grade; 2.5 percent 
named multiple programs for an individual grade. Some respondents named supplemental or other noncore 
programs. 
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Exhibit 3.6.b 






Core Reading Programs Used by Titie i Schoois, 2004 


-05 School Year 


Title 1 Schools 


Percentage of Schools in Sample that 
Use the Program 


Publisher 


Program 


1-5% 


Caron-Dellosa 


Unspecified 




DSC 


Making Meaning 




Harcourt 


Signatures 
Unspecified/Other 
Rigby Reading 




Heinemann 


Fountas Pinnel units of study 




Houghton Mifflin 


Horizons 

State Specific Edition 




McGraw-Hill 


Reading Mastery 
Spotlight on Literacy 




Saxon 


Saxon Phonics 




Scholastic 


Unspecified 
Literacy Place 




Scott Foresman 


Lectura 

Celebrate Reading 




Sophs 


Read Well 




Voyager 


Universal Literacy 




Wright Group 


Unspecified 


6-1 0% 


Harcourt 


Collections 




Houghton Mifflin 


Nation's Choice 
Reading 2005 
Legacy of Literacy 
Invitation to Literacy 
Unspecified 




McGraw-Hill 


Unspecified 




Scott Foresman 


Unspecified 
Literacy Works 


11-18% 


Harcourt 


Trophies 




McGraw-Hill 


Open Court 
Reading 




Scott Foresman 


Reading 




Success for All 


Success for All 


Source: Reading Coach Survey, Principal Survey Question Cl. 

Unweighted. 

Respondents: 273 reading coaches in Title I schools and 167 Title I principals 
Nonresponse rate: 19 percent. 

Exhibit reads: 1 percent of Title I sampled schools (reading coaches or principals) reported using a reading program 
published by Addison-Wesley for at least one of the grades K-3. The particular Addison-Wesley reading program was not 
specified. 

Notes: Programs with frequencies of less than 1 percent are presented in Appendix C, Exhibit C.3.6.b. 

The columns list the unweighted percent of schools that listed the specified reading program as the core program their school 
uses for at least one of grades K-3. 

Weights were not used because many strata did not have enough respondents to accurately figure the weighted percent. 

In many instances respondents identified a publisher but did not specify a program. These are designated here as 
“unspecified.” 

A number of publishers have developed programs tailored specifically to the needs of individual states, refen'ed to here as 
“state specific editions.” 

The Title I sample includes only Title I schoolwide project (SWP) schools with at least 40 percent of the students eligible for 
free or reduced price lunches. Therefore this information is not generalizable to all non-RE Title I schools 
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Use of Supplemental Materials 

The use of supplemental materials to support instruction in the five key elements of reading 
instruction is an important component of Reading First. Reading coaches in both RF and Title I 
schools were equally likely to report that supplemental materials used in their schools are aligned 
with SBRR. Reading coaches and principals were asked to indicate whether teachers in their 
schools used supplementary reading materials to teach reading, and also to indicate the specific 
elements of reading for which these supplementary materials were selected. 

Interestingly, although about 80 percent of the RF schools in the sample of mature RF schools 
used supplemental materials in their classrooms, reading coaches and principals reported that 
teachers in Title I schools were significantly more likely (more than 90 percent) to use 
supplemental materials than were teachers in RF schools. Further, reading coaches in Title I 
schools were more likely to report that their teachers have experience with supplemental 
materials. A statistically significant difference of approximately 10 percentage points (and in 
many cases as much as 15 percentage points) was observed across all grades and all elements of 
reading, with the exception of comprehension in kindergarten and fluency in grades 1 to 3. For 
comprehension, kindergarten teachers were equally likely to use supplemental materials in RF 
and Title I schools (35 percent vs. 37 percent). 

The fact that Title I teachers reported that they are more likely to use supplementary materials, 
and that Title I reading coaches reported that their teachers have more experience with 
supplementary materials, is puzzling. One possible explanation is that Title I schools are less 
likely to have adopted one core reading program, and instead use a number of disparate 
supplemental materials to teach various aspects of reading. Another possibility is that Title I 
schools are less likely to have a core program that emphasizes all five elements of reading, and 
may therefore be more likely to use supplemental materials for instruction in skills like decoding, 
vocabulary development, and comprehension. 

Instructional Activities and Strategies 

The RF program is, at its core, designed to have an influence on the instructional activities of the 
teachers in RF schools. In this section we describe what survey results tell us about the 
instructional activities and strategies used by the RF teachers in our sample. 



Reading coaches were asked to indicate on a scale of one to five how accurately the statement “Supplemental 
materials are aligned with SBRR” (Question HI) describes their school. The average response was 4.11 for RF 
coaches and 4.03 for reading coaches in Title I schools. This difference is not statistically significant. 

Though we tried to be clear on the surveys, it is possible that the distinctions between supplementary and 
intervention materials may not have been clear to some respondents. 

Reading coaches were asked to indicate on a scale of one to five how accurately the statement “K-3 teachers 
are experienced with supplemental reading materials” (Question HI) describes their school. The average 
response was 3.58 for reading coaches in Title I schools percent vs. 3.29 for RF coaches. This difference is 
statistically significant (p < .05). 
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Characteristics of Instruction at the Schooi Levei 



The importance of improving reading instruction is central to RF’s potential for positive effects 
in schools. As shown in Exhibit 3.7, reading coaches in RF and Title I schools were equally 
likely to report a schoolwide focus on reading and language arts and alignment of reading 
instruction with state reading and language arts standards. In contrast, however, RF 
coaches were significantly more likely than reading coaches in Title I schools to report that 
the core reading program is aligned with scientifically based reading research (4.63 vs. 
4.29). Also, consistent with the findings presented earlier, coaches in RF schools were 
significantly more likely to report involvement from the state and district in providing direction 
concerning reading instruction than coaches in Title I schools (4.22 vs. 3.66). On the other hand, 
reading coaches in RF schools were significantly less likely than their Title I counterparts to 
report that teachers in their schools are motivated to improve reading instruction (4.04 vs. 4.22). 
This difference, however, while statistically significant, is not large enough to be substantively 
meaningful. 



Exhibit 3.7 



Characteristics of Reading instruction as Reported by Reading Coaches in Mature Reading First 
and Titie i Schoois, 2004-05 Schooi Year 





Reading First 
Mean 


Title I 
Mean 


K-3 teachers are knowledgeable about scientifically based reading instruction 


3.63 


3.52 


K-3 teachers make an effort to involve parents in their children's reading 
instruction 


3.84 


3.96 


K-3 teachers are experienced with the core reading program 


3.99 


4.05 


K-3 teachers are motivated to improve reading instruction 


4.04 


4.22* 


The distriot provides direotion conoerning reading instruction 


4.12 


3.96 


The state provides direction concerning reading instruction 


4.22* 


3.66 


Reading instruotion in K-3 classrooms is aligned with the state 
reading/language arts content standards 


4.45 


4.43 


There is a schoolwide focus on reading and language arts 


4.52 


4.59 


The core reading program is aligned with SBRR 


4.63* 


4.29 



Source: Reading Coach Survey, Question HI. 

Weight: Reading coach. 

Weighted respondents: 2,215 RF reading coaches, 7,563 reading coaches in Title I schools. 

Nonresponse rates across survey items: 1 to 3 percent. 

Exhibit reads: Reading coaches were asked to indicate on a scale of one to five how accurately the statement “Teachers use a 
variety of instructional materials to fill in gaps in the core program” described their school. The average response for RF 
schools was 3.46 for RF schools and 3.69 for Title I schools. This difference is statistically significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I schools. 



Collaboration on Reading Instruction 

Reading coaches in mature RF schools reported that their schools have more time regularly set 
aside for staff to collaborate on reading lesson planning and instruction. The difference, although 
statistically significant, is not large, more often than monthly in RF schools and somewhat less 
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frequently in Title I schools. Teachers, however, corroborate this finding. As shown in Exhibit 
3.8, teachers in Title I schools were more likely to report that no time was set aside for 
collaboration on reading lesson planning and instruction (17 percent vs. 10 percent), or for 
observation of reading instruction in other classrooms (62 percent vs. 54 percent). 



Exhibit 3.8 

Teacher Reports on the Type and Frequency of Coiiaboration About Reading in Mature 



Reading First and Titie i Schoois, 2004-05 Schooi Year 

Reading First Titie i 

Teachers Teachers 

Type and Frequency of Coiiaboration: Percent Percent 

Collaborate on reading lesson planning and instruction 

...Not at all 10% 17%* 

...Infrequently (monthly or less) 43* 33 

...Once a week or more 38 35 

...Informally, as needed 9 15* 

Observe reading instruction in other classrooms 

...Not at all 54 62* 

...Infrequently (monthly or less) 24 19* 

...Once a week or more 1 1 

...Informally, as needed 21* 18 

Help with coaching or be coached about reading by other 
teacher 

...Not at all 23 39* 

...Infrequently (monthly or less) 45* 35 

...Once a week or more 12* 7 

...Informally, as needed 20 19 

Source: Teacher Survey, Question C3. 

Weight: Teacher. 

Weighted respondents: 64,545 RF teachers, 201,431 Title I teachers. 

Nonresponse rates across survey items: < 1 percent. 



Exhibit reads: 43 percent of K-3 teachers in RF schools reported having time set aside “infrequently” (monthly or less) 
for collaboration on lesson planning and instruction, compared with 33 percent of K-3 teachers in Title I schools. This 
difference is statistically significant (1 = Not at all; 2 = 1-4 times per year; 3 = 5-8 times per year; 4 = once a month; 5 = 
once a week or more) 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 
Title I respondents. 



Instructional Activities 

We created six composites to summarize teachers’ self-ratings of the centrality of a series of 
instructional activities associated with the following reading dimensions and other instructional 
features: 1) phonemic awareness and decoding; 2) vocabulary; 3) comprehension; 4) fluency; 5) 
use of scientifically based instructional strategies and materials in their classroom; and 6) 
negative alignment with scientifically based reading research. (See Appendix D for a list the 
individual items included in each composite.) Scores were computed for each composite based 
on the percentage of instructional activities specified in that composite that a teacher rated as 

Reading coaches were asked to indicate on a scale of one to five (1 = Not at all; 2 = 1^ times per year; 3 = 5-8 
times per year; 4 = once a month; 5 = once a week or more) how often time is set aside for collaboration on 
reading lesson planning. The average response for RF schools was 4.23 and 3.88 for Title I schools. This 
difference is statistically significant (p < .05). 
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“central to their instruction.” For example, if a kindergarten teacher rated six of the seven 
activities that comprise the comprehension composite as central, then their score would be 85.7 
percent. Exhibit 3.9 presents the mean percents for the six composites obtained teachers in each 
of the four target grades in mature RF schools and Title I schools. 



Exhibit 3.9 



Teacher Ratings of the Centrality of SBRR Aligned Instructional Activities in 
Mature Reading First and Title 1 Schools, 2004-05 School Year 




Reading First 


Title 1 




Teachers 


Teachers 




Mean Percent 


Mean Percent 


Phonemic Awareness and Decoding 


Kindergarten 


91 %* 


86% 


1st grade 


85* 


81 


2nd grade 


63 


59 


3rd grade 


58 


54 


Comprehension 


Kindergarten 


71 


69 


1st grade 


74 


72 


2nd grade 


67 


70 


3rd grade 


72 


70 


Vocabulary 


Kindergarten 


62 


60 


1st grade 


88 


87 


2nd grade 


71 


72 


3rd grade 


81* 


75 


Fluency 

Kindergarten 


1st grade 


87* 


83 


2nd grade 


57 


58 


3rd grade 


56* 


47 


Overall Composite SBRR 


Kindergarten 


77* 


68 


1st grade 


79* 


76 


2nd grade 


76* 


72 


3rd grade 


75* 


70 


Overall Composite Non-SBRR 


Kindergarten 


66 


67 


1st grade 


68 


70 


2nd grade 


66 


64 


3rd grade 


66 


64 



Source: Teacher Survey, Question C4. 

Weight: Teacher. 

Weighted Respondents: 38,317 RF teachers, 200,730 teachers in Title 1 schools. 

Nonresponse rates across composites and grades: 0.2 to 4.5 percent. 

Exhibit reads: Kindergarten teachers in RF schools, on average, rated 91 percent of SBRR-aligned 
phonemic awareness and decoding activities as central to their instruction; this compares with 86 percent 
for kindergarten teachers in Title I schools. This difference is statistically significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between 
RF and Title I schools. 

Kindergarten teachers were not asked any questions related to fluency instruction. 
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Across the six composites, there were some significant differences between teachers in 
mature RF and Title I schools. In general, however, the differences while statistically 
significant are not large. Specifically: 

• Kindergarten and first-grade teachers in RF schools rated a higher proportion of 
scientifically based practices to teach phonemic awareness and decoding as central 
than did teachers in Title I schools (kindergarten: 91 percent vs. 86 percent; first 
grade: 85 percent vs. 81 percent). 

• Across all four grades, there were no differences between RF teachers and teachers in 
Title I schools in the proportion of scientifically based practices to teach 
comprehension that were rated as central. 

• Third-grade teachers in RF schools rated a higher proportion of scientifically based 
practices to teach vocabulary as central than did teachers in Title I schools (81 percent 
to 75 percent). There were no significant differences between RF teachers and 
teachers in Title I schools at the grade levels. 

• First- and third-grade teachers in RF schools rated a higher proportion of 
scientifically based practices to teach fluency as central than did teachers in Title I 
schools (first grade: 87 percent to 83 percent; third grade: 56 percent vs. 47 percent). 

• RF Teachers across all four grades rated a higher proportion of scientifically based 
teaching strategies and materials as central than did teachers in Title I schools. The 
differences in grades 1, 2 and 3 are substantively small although statistically 
significant. The difference in kindergarten is larger and more substantively 
meaningful (77 percent vs. 68 percent). 

• There were no significant differences between K-3 RF teachers and teachers in Title I 
schools in the proportion of non-scientifically based activities that teachers rated as 
central to their instruction. 

Summary 

The Reading First legislation requires all aspects of reading instruction in RF schools to be 
scientifically based, and the RF guidance recommends considering the allocation of time for 
reading instruction, as well as carefully selecting scientifically based materials, activities, and 
strategies for reading instruction. On the issue of time allocation, significantly more RF schools 
reported having scheduled reading blocks than did non-RF Title I schools (98 percent vs. 92 
percent), and RF schools reported scheduling about 15 more minutes for their reading blocks in 
grades 1-3 than non-RF Title I schools. In addition, teachers in RF schools reported spending 
about 19 more minutes per day teaching reading than non-RF Title I teachers. Compared to 
teachers in non-RF Title I schools, significantly more teachers in new RF schools reported 
increasing the amount of time they spent teaching reading compared to the previous year (across 
all grades, 35 percent vs. 61 percent). 

Reading First schools have made changes to their reading programs since they received their RF 
funds. New RF schools were significantly more likely than their non-RF Title I counterparts to 
have adopted a new core reading program (39 percent vs. 16 percent), to have added new 
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intervention programs for struggling readers (74 percent vs. 43 percent), to have added new 
supplementary materials (69 percent vs. 58 percent), and to have adopted new materials for 
English Language Learners (43 percent vs. 23 percent). Reading Lirst schools also were more 
likely than non-RL Title I schools to report the involvement of districts (88 percent vs. 83 
percent) and states (31 percent vs. 18 percent) in the selection of a core reading program. The 
particular types of materials selected and used by RL and non-RL Title I schools differ as well. 
Reading Lirst schools are more likely than non-RL Title I schools to use supplemental materials 
designed especially for ELLs to support the instruction of ELL students in their classrooms (60 
percent vs. 41 percent). Non-RL Title I schools are more likely than RL schools to use 
supplemental materials to teach phonemic awareness (74 percent vs. 60 percent in kindergarten), 
decoding (79 percent vs. 63 percent in first grade), vocabulary (70 percent vs. 51 percent in first 
grade), and comprehension (75 percent vs. 60 percent in third grade). 

Reading instructional activities and strategies also appear to be aligned with the tenets of the 
Reading Lirst program. Teachers in Reading Lirst schools were more likely to report instruction 
that aligned with SBRR is central to their instruction than were teachers in Title I schools. 
Specifically, kindergarten, first- and second-grade teachers in RL schools were significantly 
more likely to report using scientifically based practices to teach decoding and phonemic 
awareness; kindergarten teachers were significantly more likely to use scientifically based 
practices to teach comprehension; and third-grade teachers were more likely to use scientifically 
based practices to teach vocabulary and fluency than were Title I teachers. In addition, RL 
kindergarten, first-, and second-grade teachers were significantly less likely than Title I teachers 
to use activities not aligned with SBRR as a central part of their reading instruction. Reading 
Lirst teachers across all four grades were more likely than Title I teachers to report using 
scientifically based teaching strategies and materials. 
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Chapter 4: Interventions for Struggling Readers 

A central objective of the Reading First legislation is to provide effective instruction to children 
in grades K-3 who are having difficulty learning to read, may be at risk of being referred to 
special education, or are having difficulty mastering the five key components of reading. 

Reading First aims to provide targeted interventions to those students who have been identified 
as struggling readers — expressly to help them overcome difficulties that could otherwise lead to 
unnecessary special education referrals. 

This section presents findings organized into several subsections: a) identification of students 
who may need interventions to develop their reading skills and the availability of intervention 
services to serve them, b) methods used to meet the needs of struggling readers, and c) 
coordination of services provided to struggling readers. 

Identification for and Availability of Interventions for Struggling 
Readers 

Principals were asked to identify which sources of information they used (during the 2004-05 
school year) to identify struggling readers in need of intervention services. In general, principals 
in RF and Title I schools reported that they drew from multiple sources to identify struggling 
readers requiring interventions. Most principals (80 percent) reported having used eight out of 
the ten sources listed in Exhibit 4.1. The most common sources were readily available reading 
tests, such as progress monitoring tests, tests from the core reading program, and diagnostic tests, 
as well as recommendations from teachers. More than 90 percent of both RF and Title I school 
principals reported using these methods in that year to identify students as struggling readers in 
need of interventions. Indeed, reading coaches in both RF and Title I schools reported using 
reading assessments to screen students for difficulties.^^ 

Principals in RF schools were more likely to rely on the results of progress monitoring and 
screening tests to identify students as struggling readers than were principals in Title I 
schools (98 percent vs. 90 percent and 87 percent vs. 82 percent, respectively; see Exhibit 

4.1). There were no significant differences between RF and Title I principals in their reported 
use of a) scores on tests from the core reading program, b) diagnostic tests, and b) standardized 
tests, to identify struggling readers. Nor were there any differences between RF and Title I 
schools’ identification of struggling readers based on teacher or other school staff 
recommendation or classroom observations. 

Reading coach recommendations were a significantly more common source of information 
used to identify struggling students in RF schools than in Title I schools (92 percent vs. 55 

Reading coaches were asked to indicate on a scale from one to five how accurately the statement “reading 
assessments are used to screen students for reading difficulties” (Question HI) describes their school. The 
average response for RF schools was 4.32 and 4.28 for Title I schools. This difference is not statistically 
significant. 
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percent); this is not surprising given that Title I sehools do not necessarily have a reading coach 
on staff. Finally, principals in Title I schools reported that they were significantly more likely to 
use parent requests to identify struggling readers than are RF principals (80 percent vs. 70 
percent). 



Exhibit 4.1 



Methods Used to identify Students for Reading interventions, in Mature Reading First and Titie i 
Schoois, 2004-05 Schooi Year 





Reading First 
Schools 


Title 1 
Schools 


Method 


Percent 


Percent 


Progress monitoring tests 


98%* 


90% 


Teacher recommendation 


97 


98 


Scores on tests frem core reading program 


95 


92 


Diagnostic tests 


91 


91 


Reading ceach recommendation ' 


92* 


55 


Standardized achievement tests 


88 


88 


Screening tests 


00 


82 


Documented ciassroom observations 


84 


85 


Other schooi staff recommendations 


76 


78 


Parent request 


71 


00 

o 



Source: Principal Survey, Question E2. 

Weight: Principal. 

Weighted respondents: 2,189 Reading First principals, 14,487 Title I principals. 

Nonresponse rates across survey items: 1.4 to 4.2 percent. 

Exhibit reads: Reading First schools are more likely than are Title I schools to use progress monitoring tests to identify students 
for reading interventions (98 percent vs. 90 percent). This difference is statistically significant (p < .05). 

1 . Only schools that have reading coaches are included in the analysis of this item. 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I schools. 



Availability of Services 

There were no substantive or statistically significant differences between RF and Title I 
schools in terms of availability of intervention services for struggling readers; about 80 
percent of principals across both RF and Title I schools reported that reading intervention 
services are available for students who need them. Nor did the average waiting time for students 
to receive services differ significantly across RF and Title I schools. In about two-thirds of the 
schools, principals reported that there was no wait time at all for students in need of intervention. 
On average, students in need of such services wait an average of about one week before 
receiving services. 

Methods to Meet the Needs of Struggling Readers 

Once students are identified as struggling readers, teachers use a variety of supports to meet their 
needs. Teachers were asked to indicate, for the previous month, which additional supports they 
used with their students (e.g., providing extra practice in key areas, working with specialists). 

Kindergarten and tbird-grade teachers in RF schools were significantly more likely than 
teachers in Title I schools to report supporting struggling readers by using materials that 
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supplement the core reading program (Exhibit 4.2, kindergarten teachers: 70 percent vs. 62 
percent; third- grade teachers: 74 percent vs. 66 percent). There were no significant differences 
for first- and second-grade teachers. RF teachers in three grades (kindergarten, second, and 
third) were significantly more likely than their counterparts in Title I schools to place their 
struggling students in intervention programs (kindergarten: 54 percent vs. 45 percent; second 
grade: 70 percent vs. 61 percent; third grade: 68 percent vs. 60 percent). 

Exhibit 4.2 also shows that RF teachers in all four grades were significantly more likely 
than Title I teachers to rely on diagnostic assessments to determine their struggling 
readers’ core deficits (kindergarten: 75 percent vs. 59 percent; first grade: 78 percent vs. 69 
percent; second grade: 72 percent vs. 65 percent; third grade: 70 percent vs. 61 percent). 
Relatively few teachers across both RF and Title I schools reported using reading specialists, 
placing their students in a separate core reading program, or providing reading instruction in 
ELL students’ home language to meet the needs of their struggling students. 
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Exhibit 4.2 



Teachers’ Use of Supports Used to Meet the Needs of Struggiing Readers in Mature Reading First 
and Titie i Schoois, 2004-05 Schooi Year 



staff Activities 

Diagnostic assessment to determine core deficits 
Work with tutor on one-to-one basis 
Work with reading specialist on one-to-one basis 
Work with reading specialist in small group 
Work with more advanced peer 




in phonemic awareness 
in decoding 
in comprehension 
in fluency 
Extra instructional time 



Placement in materials that supplement the core 

reading program 

Placement in different level of core reading prograrr 
Placement in separate core reading prograrr 
Placement in special inten/ention prograrr 
Special materials for parents to provide practice 





0 20 



40 60 80 



. __ Statistically 
' Significant at 



Percent of Teachers 



p < .05 



Ranges Across Grades K-3 



Source: Teacher Survey, Question C9. 

Weight: Teacher. 

Weighted respondents: 38,309 RF teachers, 201,864 Title 1 teachers. 

Nonresponse rates across survey items and grades: 0.1 to 1.1 percent. 

Exhibit reads: Across three grades, RF teachers were significantly more likely than Title 1 teachers to use diagnostic assessments to 
determine core deficits. 

Notes: For the items under ‘Support for ELLs’ the grade level means — as represented by the bars — are based on teachers who 
reported having ELL students in their classrooms. 

Each bar represents the range of the grade-level means computed for each item. 

For each grade level, a significant difference (p < .05) between RF teachers and Title 1 teachers is indicated by the grade level 
symbol (i.e., K, 1, 2, 3). For example, if kindergarten, second grade, and third grade showed significant differences between RF 
teachers and Title 1 teachers, then ”K23” would be presented. 

See Appendix C, Exhibit C.4.2 for additional statistics. 
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In both RF and Title I schools, most teachers reported providing extra practice in one of the key 
dimensions of reading to meet the needs of their struggling readers. We constructed a composite 
measure to summarize teachers’ provision of extra practice that includes the three dimensions of 
reading in which struggling readers most typically need additional practice, phonemic awareness, 
decoding, and fluency. Scores on the composite range from 0 to 3; a score of 0 indicates that the 
teacher did not provide extra practice in any of these three dimensions in the last month, while a 
score of 3 means that the teacher provided extra practice in all three dimensions. Exhibit 4.3 
shows the grade-level mean scores on the composite measure of extra practice teacher reported 
providing to students last month. Across three grades — kindergarten, second and third — on 
average, teachers in RF schools reported providing practice in significantly more 
dimensions than did teachers in Title I schools; however, these differences are substantively 
small. 



Exhibit 4.3 



Teachers’ Scores on Composite Measure Extra Practice Provided to Struggiing Readers in 
the Previous Month, in Mature Reading First and Titie i Schoois, 2004-05 Schooi Year 



Grade 


Reading First Teachers 


Title 1 Teachers 


Mean 


Mean 


Kindergarten 


2.75* 


2.66 


1st grade 


2.83 


2.78 


2nd grade 


2.83* 


2.74 


3rd grade 


2.72* 


2.45 



Source: Teacher Survey, Question C9. 

Weight: Teacher. 

Weighted respondents: 38,386 RF teachers, 203,540 Title I teachers. 

Nonresponse rates across survey items and grades: 0.3 to 0.8 percent. 

Exhibit reads: Reading First teachers in grade 3 provided significantly more practice in more of the key dimensions of 
reading than did Title I teachers to struggling readers (2.72 vs. 2.45, p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between teachers in RF and 
Title I schools. 



Principals and reading coaches were also asked to identify the materials and staff activities used 
in their school to meet the needs of struggling readers. As seen in Exhibit 4.4, about 90 percent 
of principals in RE and Title I schools reported having used a core reading program, augmented 
by supplemental materials during the 2004-05 school year to meet the needs of struggling 
readers. Principals in Title I schools were significantly more likely to report that their 
schools used separate program materials for interventions than were principals in RF 
schools (78 percent vs. 70 percent). Further, reading coaches in Title I schools were more 
likely to report that their teachers are experienced with reading intervention materials than were 
reading coaches in RF schools. However, RF coaches were more likely to agree that reading 
intervention materials are aligned with scientifically based research than reading coaches in 
Title I schools. When asked about the prior school year, RF school staff were significantly more 
likely than staff in Title I schools to have reported using only the materials from their core 
reading programs to meet the needs of struggling readers (27 percent vs. 20 percent). 
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These findings suggest that in general, teachers rely heavily on the core reading program and 
supplementary materials. Because core reading programs are often chosen to emphasize 
research-based, systematic instruction, relying on them may indicate that teachers are adopting a 
systematic approach to teaching reading. This is in contrast to using reading instructional 
activities from a variety of sources (e.g., several different reading programs), which could lead to 
less systematic instruction. 



Exhibit 4.4 



Materiais Used to Meet the Needs of Struggiing Readers in the Last Year, as Reported by the 
Principais or Reading Coaches in Reading First and Titie i Schoois, 2004-05 Schooi Year 



Materials 


RF Schools 
Percent 


Title 1 Schools 
Percent 


Use core reading program with suppiementai materiais 


90% 


90% 


Use separate program materiais in interventions 


70 


78* 


Use aiternative materiais designed for Engiish iearners 


47 


49 


Use reading materiais written in students’ home ianguage 


32 


36 


Use core reading program oniy 


27* 


20 



Source: Principal Survey, Question El; Reading Coach Survey, Question El. 

Weight: School. 

Weighted respondents: 3,893 RE principals or reading coaches; 14,171Title I principals or reading coaches. 

Nonresponse rates across survey items and respondents: 0.7 to 7.5 percent. 

Exhibit reads: Reading Eirst schools were significantly less likely than Title I schools to use separate program materials to meet 
the needs of struggling readers (70 and. 78 percent, p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RE and Title I schools. 



Principals and reading coaches reported using a variety of staff and activities to meet the needs 
of struggling readers. As seen in Exhibit 4.5, across both RF and Title I schools, classroom 
teachers and trained aides were most often used to provide additional direct instruction 
and practice opportunities for struggling readers during the school day. Overall, the 
patterns were quite similar for both RF and Title I schools. However, certified special 
education teachers were significantly more likely to be called upon to make recommendations 
about how to meet the needs of these students in Title I schools than in RF schools (83 percent 
vs. 72 percent). Title I schools were also significantly more likely than RF schools to use 
untrained aides during the school day to assist in meeting the needs of struggling readers (42 
percent vs. 34 percent). 
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Exhibit 4.5 



Staff Activities Used to Meet the Needs of Struggiing Readers in the Previous Year, as Reported 
by the Principais or Reading Coaches in Reading First and Titie i Schoois, 2004-05 Schooi Year 





RF Schools 


Title 1 Schools 


Staff Activities 


Percent 


Percent 


Classroom teacher provides additional practice opportunities 


98% 


98% 


Classroom teacher provides additional direct instruction 


97 


97 


Trained aides or volunteers work with students during class 


87 


87 


A certified special education teacher provides recommendations on 
accommodations 


72 


00 

GO 


A certified reading specialist works directly with students 


51 


57 


Trained aides or volunteers work with students before or after school 


51 


52 


A certified bilingual/ESL teacher provides recommendations on 
accommodations 


44 


43 


Untrained aides or volunteers work with students during class 


34 


42* 


Untrained aides or volunteers work with students before or after school 


15 


17 


Source: Principal Survey, Question El; Reading Coach Survey, Question El. 
Weight: School. 



Weighted respondents: 3,893 RF principals or reading coaches; 14,171 Title 1 principals or reading coaches. 

Nonresponse rates across survey items: 0.3 to 3.1 percent. 

Exhibit reads: Almost all (98 percent) of Reading First and Title 1 teachers provide additional practice for struggling readers. 
There is no statistically significant difference on this item between Reading First and Title 1 teachers. 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between teachers in RF and 
Title 1 schools. 



Time Set Aside for Coordination of Interventions Among Teachers 
and Other Staff 

Reading intervention efforts with struggling readers or ELL students, to be successful, must 
allow time for teachers, ELL staff, or special education teachers to coordinate their instructional 
activities. Absent such coordination — if teachers and other staff are not working on the same 
reading subskills and in the same sequence — the usefulness of the reading intervention is likely 
to be compromised. 

As seen in Exhibit 4.6, in general, relatively few teachers in any grade reported that time is set 
aside to coordinate instructional activities; less than 15 percent of teachers in both RE and Title I 
schools reported that time was available weekly. Although more teachers at all grades reported 
that time is set aside for meeting with ELL and special education staff (in both RE and Title I 
schools), over 31 percent (and up to 45 percent) reported that no such time was designated. 
Specifically, RF kindergarten, first- and second-grade teachers were significantly more 
likely than Title I teachers to report no time set aside for coordination of the reading 
instruction provided to their special education students (kindergarten: 51 percent vs. 44 
percent, first-grade: 42 percent vs. 35 percent; second grade: 37 percent vs. 30 percent). 
There were no significant differences between RF and Title I teachers with regard to time 
set aside to coordinate instruction with ELL staff (Exhibit 4.6). These findings are consistent 
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with reading coach and principal reports of how often time is set aside for coordination between 
teachers, ELL staff and staff providing special education services (SPED) to students. 



Exhibit 4.6 



Amount of Time Teachers in Reading First and Titie i Schoois Set Aside to Coordinate 
interventions with Staff, 2004-05 Schooi Year 



Teachers that Coordinate Readings 
interventions with Speciai Education Staff 
in RF and Titie i Schoois 



Key 

I 1 Reading First 

• . . • Title I 



Teachers that Coordinate Reading 
Interventions with ELL Staff in RF and Title I 
Schools Staff 



K12 



Not at all 



Infrequently 



Weekly 



1 



♦ ♦ 



Informally 



Statistically 100 80 60 40 20 0 0 20 40 60 80 100 

Significant at Percent Percent 

p < .05 

Source: Teacher Survey, Questions C3F and C3G. 

Weight: Teacher. 

Weighted respndents: special education coordination — 64,457 RF teachers, 201,054 Title I teachers; ELL coordination — 37,641 
RF teachers, 94,880 Title I teachers. 

Nonresponse rates across survey items and grades: < 1 percent. 

Exhibit reads: In three of four grades more Reading First teachers than Title I teachers indicated that there is no time set aside for 
coordinating reading interventions for struggling readers. 

Notes: Grade-level means for coordinating interventions with ELL staff (right panel) are based on teachers who reported having 
ELL students in their classrooms 

Each bar represents the range of the grade-level means computed for each item. 

For each grade level, a significant difference (p < .05) between RF teachers and Title I teachers is indicated by the grade level 
symbol (i.e., K, 1, 2, 3). For example, if kindergarten, second grade, and third grade showed significant differences between RF 
teachers and Title I teachers, then ”K23” would be presented. 

See Appendix C, Exhibit C.4.6 for additional statistics. 



Principals and reading coaches were asked how often during the year time is regularly set aside for coordination 
with SPED and ELL staff, on a scale from “not at all” to “once a week or more” (Question Dl). The average 
response regarding coordination with SPED staff for RE schools was 3.16 and 3.05 for Title I schools, which 
corresponds to between five to eight times per year on the scale. The average response regarding coordination 
with ELL staff (in schools serving ELL students) was 2.79 in RE schools and 2.58 in Title I schools, 
corresponding to between one to four times and five to eight times per year. Neither of these differences are 
statistically significant. 
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Summary 



One of the goals of Reading First is for schools to identify, provide, and coordinate services to 
students who are struggling to learn to read. Staff in both RF and Title I schools reported using 
information from readily available reading tests to identify students as struggling readers. 
However, RF teachers across all grades were significantly more likely to rely on diagnostic 
assessments to determine their struggling readers’ core deficits than were teachers in 
corresponding grades in Title I schools. Additionally, there were differences in schools’ reported 
use of staff recommendations to identify struggling readers. While both RF and Title I schools 
reported using teacher recommendations quite often to identify students for interventions, RF 
schools more often relied on reading coach recommendations to identify struggling readers than 
Title I schools. 

There were only small differences in RF and Title I schools’ use of materials and staff activities 
to meet the needs of struggling readers. In general, teachers reported relying heavily on the core 
reading program and supplementary materials. In addition, teachers in both RF and Title I 
schools provided extra practice opportunities in the key dimensions of reading to meet the needs 
of struggling readers. However third- grade teachers in RF schools reported providing their 
struggling readers with extra practice opportunities in phonemic awareness, decoding, and 
fluency more often than their counterparts in Title I schools. More generally, RF teachers in 
three of four grades reported providing practice in significantly more reading dimensions than 
did Title I teachers. 

RF and Title I schools are similar with respect to planning and coordinating instruction for 
struggling readers; in general, teachers in both types of schools reported that little time is set 
aside to coordinate interventions for struggling readers with ELL or special education staff. This 
finding is consistent with reading coach and principal reports from both sets of schools, 
indicating that this sort of coordination, while important to the success of intervention services, 
has yet to be achieved. 
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Assessment of students’ reading proficiency is a central element of Reading First; the legislation 
specifically requires that schools assess students for screening, diagnostic, progress monitoring, 
and outcome purposes in the five core dimensions of reading instruction (i.e., phonemic 
awareness, phonics, vocabulary, reading comprehension, and fluency). States and districts are to 
provide assistance to RF schools in selecting, administering, and interpreting reading 
assessments as well as professional development to teachers in the use of reading assessments, 
particularly with students at risk of reading failure. The Reading First program does not advocate 
the use of any specific assessment but rather requires that reading assessments selected by states, 
districts, or schools be psychometrically strong and aligned with instruction (U.S. Department of 
Education, OESE, “Guidance for the Reading Eirst Program,” April 2002). 

In this section we describe differences between Reading Eirst schools and non-RE Title I schools 
in (a) the selection and interpretation of reading assessments and (b) the types of reading 
assessments teachers find useful in their classroom applications. 

Selection and Interpretation of Reading Assessments 

Significantly more Reading First principals reported that their schools received assistance 
selecting assessment instruments than did non-RF Title I principals (76 percent vs. 56 
percent) and interpreting assessment results (82 percent vs. 70 percent). (Exhibit 5.1) 



Exhibit 5.1 

Assistance for K-3 Reading Assessment Activities in Mature Reading First and Titie i Schoois, 




Selecting assessment instruments Interpreting assessment results 



Source: Principal Survey, Questions B7b and B7f. 

Weight: Principal. 

Weighted respondents: 2,169 RF principals; 14,481 Title I principals. 

Nonresponse rates across survey items: 1.5 to 2.3 percent. 

Exhibit reads: 76 percent of RF schools received assistance selecting assessment instruments, as compared with 56 percent of 
Title I schools. This difference was statistically significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
schools. 

See Appendix C, Exhibit C.5.1 for additional statistics. 
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Selection of Reading Assessments 

As shown in Exhibit 5.2, over three-quarters of the principals in RF and Title I schools 
identified the district as responsible for the selection of assessment instruments. However, 
half of Reading First principals (51 percent) indicated that selecting assessment 
instruments was also the state’s responsibility, compared to only 31 percent of Title I 
principals.^^ 

The direct role that states and districts played in selecting reading assessments for Reading First 
schools was also evident through our review of state Reading First grant applications and 
interviews with state Reading First coordinators. Of the states and jurisdictions reporting, 55 
percent (29 states) selected all of the assessments to be used administered in Reading First 
schools. An additional 18 states (32 percent) selected at least one of the assessments (but not all) 
to be administered as part of a portfolio of assessments selected by the district. Only six states 
(11 percent) did not specify any required assessment. 



Exhibit 5.2 

Responsibiiity for Reading Assessment Activities in Mature Reading First and Titie i Schoois 
as Reported by Principais, 2004-05 Schooi Year 

Percent 
of Schools 




State Principal District Reading 

Coach 

Selecting Assessment Instruments Interpreting Assessment Results 



□ Reading First 

□ Title I 



Source: Principal Survey, Questions D4d and D5d. 

Weight: Principal. 

Weighted respondents: 2,196 RF principals; 14,353 Title I principals. 

Nonresponse rates across survey items and respondents: 0.6 to 3.9 percent. 

Exhibit reads: 51 percent of RF principals identified the state as responsible for selecting assessment instruments, as 
compared with 30 percent of Title I principals. This difference was statistically significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
schools. 

See Appendix C, Exhibit C.5.2 for additional statistics. 



Principals could identify one or more entities as having responsibility. 
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Interpretation of Assessment Results 



Nearly all principals, about 90 percent, in Reading First and Title I schools identified 
themselves as responsible for interpreting assessment results. In addition, more than half the 
Reading First and Title I principals identified the district as responsible for this activity (57 
percent vs. 59 percent). However, almost all RF principals (93 percent) also identified the 
school’s reading coach as responsible for interpreting assessment results, compared to only half 
of Title I principals (52 percent). This significant difference should not be surprising given that, 
in contrast to Reading First schools. Title I schools were much less likely to have a designated 
reading coach. 

Teachers in Reading First schools reported having had more time to use assessment data to 
plan instruction than did teachers in Title I schools (Exhibit 5.3). For example, Reading First 
teachers were significantly more likely to report having time set aside for this task on a monthly 
basis than were Title I teachers (21 percent vs. 14 percent). In contrast. Title I teachers were 
more significantly likely to report that formal time was not set aside at all than were RF teachers 
(12 percent vs. 6 percent). These findings are consistent with principal and reading coach reports 
that teachers in RF schools have significantly more formal time set aside to use assessment data 
to plan instruction than teachers in Title I schools. 



Exhibit 5.3 



Reguiariy Scheduied and Formai Time Set Aside for K-3, Teachers to Use 
Assessment Data to Pian instruction for Mature Reading First and Titie i Schoois, 
2004-05 Schooi Year 



Frequency of Time Set Aside 


Reading First 
Teachers 
Percent 


Title 1 Teachers 
Percent 


Once a week or more 


27% 


24% 


Once a month 


21* 


14 


5-8 times 


12* 


9 


1-4 times 


24 


27* 


Not at ali 


6 


12* 


Informaiiy, oniy as needed 


10 


13* 



Source: Teacher Survey, Question C3c. 

Weight: Teacher. 

Weighted respondents: 38,002 RF teachers; 197,316 Title I teachers. 

Nonresponse rate: 2.9 percent. 

Exhibit reads: 27 percent of RF teachers had regularly scheduled and formal time set aside once a week or 
more for grade-level teachers to use assessment data to plan instruction, as compared with 24 percent of 
Title I teachers. The difference was not statistically significant (p = .184). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF 
and Title I respondents. 



Principals and reading coaches were asked to rate the frequency of an activity using a five-point scale (1 = not 
at all, 2 = 1-4 times a year, 3 = 5-8 times a year, 4 = once a month, 5 = once a week or more). The average 
response in RF schools was 3.88, corresponding most closely to “once a month,” whereas the average response 
for Title I schools was 3.38 or about 5-8 times a year. This difference is statistically significant (p < .05). 
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Types of Reading Assessments Teachers Find Useful 



Reading First funds are to be used, in part, to assess students’ reading. Reading First specifically 
requires schools to assess students’ reading for screening, diagnostic, progress monitoring, and 
outcome purposes. We asked Reading First and Title I teachers to indicate which assessments 
(formal or informal) they found useful for each of the following purposes: (a) placement or 
grouping of students, (b) determining student mastery of skills, and (c) identifying the core 
deficits of struggling readers. For each purpose, we asked teachers to name up to three 
assessments. 

We grouped their responses into the following broad categories: 

• Formal assessments: 

o Core or supplementary reading program assessments 
o District assessments^^ 
o Standardized assessments 

o State-specific assessments (i.e., test specific to a particular state) 

• Informal Assessments (e.g., running records, classroom-based assessments). 

Most teachers in both RF and Title I schools named at least one assessment that they found 
useful in placing or grouping students (90 percent), determining student mastery of skills 
(89 percent), and identifying the core deficits of struggling students (85 percent). 

Types of Assessments 

There were significant differences between RF and Title I teachers in the types of assessments 
they reported as useful for the purposes of placing or grouping students, determining student 
mastery of skills, or identifying the core deficits of struggling readers (Exhibit 5.4). For 
example, for placing or grouping students into appropriate reading groups, significantly more RF 
teachers identified standardized assessments than did Title I teachers (50 percent vs. 44 percent). 
RF teachers also were more likely to report that assessments from the core or supplemental 
reading program were useful for placing or grouping students than were Title I teachers (35 
percent vs. 27 percent). In contrast, significantly more Title I teachers identified informal 
assessments as useful than did than RF teachers for placing or grouping students (36 percent vs. 
27 percent). As shown in Exhibit 5.4, similar differences were observed for a) the purposes of 
determining student mastery of skills and b) identifying the core deficits of struggling readers. 



In order to be grouped as a “district assessment,” the teacher had to clearly label it as a district test. 
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Exhibit 5.4 

Types of Assessments (Formai and informai) Reported as Usefui by Mature Reading 
First and Titie i K-3 Teachers, by Assessment Purpose, 2004-05 Schooi Year 




Reading First 


Title 1 


Assessment Purpose 


Teachers 


Teachers 


Type of Assessment 






Placing or grouping of students 


Percent 


Percent 


Formal assessments 






Core or supplementary program assessment 


35%* 


27% 


District assessment 


4 


5 


Standardized assessment 


50’* 


44 


State-specific assessment 


15 


13 


Informal assessments 


27 


36* 


Determining student mastery of skilis 


Formal assessments 






Core or supplementary program assessment 


50* 


38 


District assessment 


5 


4 


Standardized assessment 


33 


30 


State-specific assessment 


14 


12 


Informal assessments 


28 


38* 


Identifying the core deficits of struggiing students 


Formal assessments 






Core or supplementary program assessment 


37* 


25 


District assessment 


4 


3 


Standardized assessment 


40* 


34 


State-specific assessment 


14 


14 


Informal assessments 


25 


37* 



Source: Teacher Survey, Questions C7a-C7c. 

Weight: Teacher. 

Weighted respondents: 8,367 RF teachers; 43,898 Title I teachers. 

Nonresponse rates across survey items: 10 to 15 percent. 

“ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to 
three assessments per purpose. If a teacher listed the same assessment more than once for the same purpose, 
then that assessment was counted only once (e.g., if the DIB ELS Nonsense Word Fluency test and the DIBELS 
Phonemic Segmentation Fluency test were used to place or group students, then the DIBELS was counted only 
once for that purpose). 

We were not able to categorize 35-40 percent of teachers’ responses. “Not able to categorize” includes 
responses that were too vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, 
decoding, rubric, pretest, quarterly test, skill test, fluency); responses that accounted for less than 1 percent of 
the total frequency (e.g., MGLS, parent reading, KIA); and responses that were not discemable (e.g., ELLA 
stands for Early Learning Literacy in Arkansas or the English Language and Literacy Assessment). 

In order to be grouped as a “district assessment,” the teacher had to clearly label it as a district test. 

Exhibit reads: 35 percent of RF teachers reported that they found core or supplementary reading program 
assessments useful for the purpose of placing or grouping students, as compared with 27 percent of Title I 
teachers. This difference was statistically significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF 
and Title I respondents. 

Additional, grade-level statistics are presented in Appendix C, Exhibits C.5.3a-d and C.5.4a-d. 
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Specific Assessments 



Within the broad types of assessments (as discussed above), several specific assessment 
instruments were each reported as useful instruments by at least 5 percent of all RF and Title I 
teachers: 

Formal Assessments: 

• Clay Observational Survey, 

• Dynamic Indicators of Basic Early Literacy Skills (DIB ELS), 

• Developmental Reading Assessment (DRA), and 

• STAR Reading. 

Informal Assessments: 

• Classroom-based assessments (e.g., teacher made tests), and 

• Running records or miscue analysis. 

While most teachers identified at least one assessment as useful for various instructional 
purposes, there were significant differences between RE and Title I teachers in the specific 
assessments they identified as useful (Exhibit 5.5). Across all uses of assessments, RF teachers 
were significantly more likely to name the DIBELS as useful than were Title I teachers 
(e.g., for grouping students: 39 percent vs. 10 percent). In contrast, significantly more 
Title I teachers reported that informal assessments such as classroom-based assessments 
(e.g., teacher-made tests) were useful in identifying core deficits of struggling readers than 
RF teachers (24 percent vs. 17 percent). Additionally, across all assessment uses. Title I 
teachers were significantly more likely to name formal assessments including the 
Developmental Reading Assessment (DRA), STAR Reading, and Clay Observational 
Survey than were RF teachers. As an example. Title I teachers also were more likely to report 
the DRA were useful in grouping students than were RF teachers (20 percent vs. 1 1 percent). 

In addition to these general findings, other patterns emerged in the reported usefulness of 
assessments (Exhibits 5.4 and 5.5): 

Differences by assessment purpose: 

• A greater percentage of RF and Title I teachers identified standardized assessments 
as useful for the purpose of placing or grouping students (50 percent vs. 44 percent) 
than for the purposes of identifying the core deficits of struggling readers (40 percent 
vs. 34 percent) or for determining student mastery of skills (33 percent vs. 30 
percent). This trend was also evident for the DRA and STAR Reading for both RF 
and Title I teachers, and for the DIBELS among Reading First teachers only. 

• RF and Title I teachers more often identified assessments from the core or 
supplementary reading program as useful for determining student mastery of .skills 
(50 percent vs. 38 percent) than for grouping students (35 percent vs. 27 percent) or 
for identifying the core deficits of struggling readers (37 percent vs. 25 percent). This 
trend was also evident for classroom based assessments for both RF and Title I 
teachers. 
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Exhibit 5.5 



Types of Spec/T/c Assessments (Formai and informai) Reported as Usefui by Mature 
Reading First and Titie i K-3 Teachers, by Assessment Purpose, 2004-05 Schooi Year 



Assessment Purpose 

Type of Assessment 


Reading First Teachers 


Title 1 Teachers 


Percent 


Percent 


Placing or grouping of students 


Formal assessments 






Clay Observational Survey 


5% 


8%* 


DIBELS 


39* 


10 


DRA 


11 


20* 


STAR Reading 


6 


12* 


Informal assessments 






Classroom-based assessment 


18 


20 



Running records or miscue analysis ^ 

Determining student mastery of skills 

Formal assessments 



Clay Observational Survey 4 9* 

DIBELS 26* 7 

DRA 5 12* 

STAR Reading 3 6* 

Infermal assessments 

Classroom-based assessment 24* 29 

Running records er miscue analysis 6 12* 



Identifying the core deficits of struggling students 



Formal assessments 

Clay Observational Survey 5 10* 

DIBELS 33* 10 

DRA 7 13* 

STAR Reading 2 6* 

Infermal assessments 

Classroom-based assessment 1 7* 24 

Running records er miscue analysis 17 



Source: Teacher Survey, Questions C7a-C7c. 

Weight: Teacher. 

Weighted respondents: 8,359 RF teachers; 42,724 Title I teachers. 
Nonresponse rates across survey items: 10 to 15 percent. 



Only individual assessments that constituted more than 5 percent of responses are included in this table. 

Percentages by grade or assessment purpose will not total 100 percent. Teachers were asked to list up to three 
assessments per purpose. If a teacher listed the same assessment more than once for the same purpose, then the 
assessment was counted only once. 

The category of “Classroom-based assessment” includes informal, teacher-guided assessments, such as tests created 
by the teacher, teacher observations of students, and teacher conferences with students. 

The category of “Running records or miscue analysis” represents responses in which teachers did not specify 
anything in addition to “running record” or “miscue analysis.” Instances in which teachers responded “Running 
record/(core reading program name)” were coded as “Core or supplementary reading program” as running records 
were presented as a component of the core program. 

Exhibit reads: 5 percent of RF teachers identified the Clay Observational Survey as useful for placing or grouping 
students, as compared with 8 percent of Title I teachers. This difference was statistically significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
respondents. 
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Differences by grade level 

• A greater percentage of RF and Title I kindergarten teachers than first-, second-, and 
third-grade teachers identified classroom-based assessments as useful reading 
assessment instruments. For example, for grouping students, 26 percent of RF 
kindergarten teachers and 32 percent of Title I kindergarten teachers found 
classroom-based assessments as useful, as compared to 15-18 percent of RF teachers 
in grades 1-3 and 16-19 percent of Title I teachers in grades 1-3. 

• The percentage of RF and Title I teachers who identified the DIB ELS as a useful reading 
assessment generally decreased with grade progression. As an example, for determining 
student mastery of skills, 33 percent of RF kindergarten teachers reported the DIB ELS as 
useful, which fell to 30 percent in grade 1, to 23 percent in grade 2, and to 19 percent in 
grade 3. 

• Similarly, the percentage of RE and Title I teachers who identified the usefulness of 
informal assessments also decreased with grade progression. Eor example, for placing or 
grouping students, 33 percent of RE kindergarten teachers reported classroom-based 
assessments useful, which fell to 29 percent in grade 1, to 25 percent in grade 2, and to 24 
percent in grade 3. 

Classroom Application of Reading Assessment Results 

Across grade levels, more than three-quarters of teachers in mature Reading First schools 
reported using assessment-related teaching strategies that reflect key tenets of the Reading 
First policy (Exhibit 5.6). In addition, significantly more Reading First K-3 teachers than 
Title I teachers reported each of these strategies as central to their reading instruction: 

• Using test results to organize instructional groups (83 percent vs. 73 percent); 

• Using tests to determine progress on skills (85 percent vs. 78 percent); and 

• Using diagnostic tests to identify students who need reading intervention services 

(75 percent vs. 65 percent). 

As shown in Exhibit 5.6, the use of the four assessment strategies that are not the emphasis of 
Reading Eirst policy was less widespread. Title I teachers reported using informal reading 
inventories as a central part of their reading instruction at significantly higher rates than did 
Reading Eirst teachers (61 percent vs. 56 percent). In contrast, RE teachers reported using tests to 
determine who can benefit from the core reading series at higher rates than did Title I teachers 
(54 percent vs. 45 percent). 



See Appendix C Exhibits C.5.4a-d and C.5.5a-d for grade-level statistics. 
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Exhibit 5.6 



Teachers’ Use of Assessments for Mature Reading First and Titie i Schoois, 2004-05 Schooi 
Year 



Assessment Strategy 


Reading First 
Teachers 


Title 1 
Teachers 


Percent 


Percent 


Reading First Policy 


Use test results to organize instructional groups 


83%* 


73% 


Use tests to determine progress on skills 


85* 


78 


Use diagnostic tests to Identify students who need reading intervention 
services 


75* 


65 


Non Reading First Policy 






Use informal reading inventories 


56 


61* 


Use tests to determine who can benefit from the core reading series 


54* 


45 


Use screening tests to identify students who need a supplementary reading 
program 


51 


48 


Conduct miscue analysis, analyzing errors students make while reading 
aloud 


50 


53 



Source: Teacher Survey, Question C6. 

Weight: Teacher. 

Weighted respondents: 38,408 RF teachers; 202,772 Title I teachers. 

Nonresponse rate: 0.6 to 1.3 percent. 

Exhibit reads: 83 percent of RF teachers reported using test results to organize instructional groups as a central part of their 
reading instruction, as compared with 73 percent of Title I teachers. This difference was statistically significant (p < .05). 
Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
respondents. 



Interestingly, even though RF teachers were more likely to report using assessments to monitor 
skill progress than were Title I teachers (85 percent vs. 78 percent), there were no differences in 
reading coaches reporting of the use of reading assessments for this purpose (4.51 and 4.41^^). 
These findings may indicate that while school leadership in both RF and Title I schools 
understand the importance of using assessment data to monitor students. Title I teachers have not 
implemented this strategy to the same degree as RF teachers. 

Of important note, when looking at the total population of Reading First schools, similar 
percentages of kindergarten, first-, and second-grade teachers in new and mature Reading 
First schools reported the central use of assessment-related teaching strategies. However, a 
significantly greater percentage of third-grade teachers in new Reading First schools than mature 
Reading First schools reported the use of informal reading inventories (62 percent vs. 53 percent) 
or miscue analysis (60 percent vs. 52 percent) as a central component of their reading instruction; 
these types of informal assessments are not the emphasis of Reading First. 



Reading coaches were asked to rate on a five-point scale how accurately the statement “Reading assessments 
are used to monitor student progress” describes their school. The average response was 4.5 1 for RF schools and 
4.41 for Title I schools. This difference is not statistically significant. 
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Summary 



Principals and teachers in both RF and in non-RF Title I schools indicated that assessment plays 
an important role in their reading programs. At the same time, there are some significant 
differences between RF and Title I schools. Most Reading First schools — significantly more than 
non-RF Title I schools — received external help in selecting assessment instruments (76 percent 
vs. 56 percent) and interpreting assessment results (82 percent vs. 70 percent). Reading First and 
Title I principals were equally likely to report that the district was responsible for selecting 
assessment instruments (about 80 percent). However, Reading First principals were significantly 
more likely to also identify the state as responsible for this selection than were principals in 
Title I schools (51 percent vs. 30 percent). Most RF and Title I principals identified themselves 
as responsible for interpreting assessment results (90 percent). However, in RF schools, reading 
coaches were signifieantly more likely to be involved in interpreting test results than were 
coaches in Title I schools (92 percent vs. 51 percent). Further, across grade levels, significantly 
more Reading First teachers reported having had regular time set aside at least once a month to 
use assessment data to plan instruction than did teachers in Title I schools (48 percent vs. 38 
percent). 

Most teachers in both RF and Title I schools named at least one assessment that they found 
useful in placing or grouping students (90 percent), determining student mastery of skills (89 
percent), and identifying the core deficits of struggling students (85 percent). There were 
significant differences, however, in the types of assessments teachers identified as useful. In 
general, RF teachers were more likely to report formal assessments, whereas Title I teachers 
named informal assessments. For each assessment purpose, RF teachers were more likely than 
Title I teachers to identify assessments from the core or supplementary reading program (e.g., for 
determining student mastery of skills, 50 percent vs. 38 percent)*^^ and standardized tests (e.g., 
for placing or grouping students, 50 percent vs. 44 percent), such as the DIBELS (e.g., for 
identifying the core deficits of struggling students, 33 percent vs. 10 percent). 

In contrast, across the three uses of assessments. Title I teachers were significantly more likely to 
identify informal assessments as useful than were RF teachers (e.g., for determining student 
mastery of skills, 38 percent vs. 28 percent). Specifically, Title I teachers were more likely to 
identify classroom-based assessments as useful for identifying the core deficits of struggling 
students than were RF teachers (24 percent vs. 17 percent); a similar pattern holds for running 
records or miscue analysis (e.g., for placing or grouping students, 19 percent vs. 10 percent). 

Although most RF and Title I teachers were able to identify at least one assessment as useful 
across purposes, significantly more RF teachers than non-RF Title I teachers, reported using 
assessment data as a central component of their classroom reading instruction to organize 
instructional groups (83 percent vs. 73 percent), determine progress on skills (85 percent vs. 78 
percent), and identify students who need reading intervention services (75 percent vs. 65 
percent). 



For space and clarity, percentages are reported for only one purpose per assessment. 
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Chapter 6: Oversight and Classroom Support 
Activities 



Although it is teachers who ultimately deliver reading instruction, a number of other 
personnel are often involved in oversight or support of teachers’ reading instruction. One 
model for supporting teachers’ reading instruction involves hiring a reading coach who will 
help teachers develop the skills needed to implement reading instruction that is aligned with 
scientifically based reading instruction. Others who are often involved in oversight of a 
school’s reading program include school principals, district staff and state personnel. This 
section first reports the findings regarding the responsibilities and coaching activities of 
reading coaches, followed by subsections discussing responsibility for oversight of the 
reading program, the role of the school principal, and state monitoring and evaluation. 

Reading Coaches 

Almost all states require Reading First schools to have reading coordinators to assist in the 
implementation of their reading programs. In many schools, this person is called a reading 
coach and is responsible for working with teachers so that they can implement reading 
activities aligned with SBRR. 

According to school principals, 98 percent of Reading First schools (new and mature) have a 
person designated as a reading coach. RF reading coaches reported spending an average of 87 
percent of their time as a reading coach for grades K-3 in the sampled school. Sixty-two 
percent of reading coaches in RF schools reported that their duties as K-3 reading coach in 
the sampled school constituted a full-time job. Reading coaches worked with an average of 
1.2 schools (89 percent work with only one school), and with 21 teachers (83 percent work 
with more than 10 teachers). 

Mature Reading First schools were significantly more likely than Title I schools to report 
having a reading coach on staff (98 percent vs. 88 percent). As shown in Exhibit 6.1, reading 
coaches"^^ in Reading First and Title I schools work with the same number of schools and 
teachers. However, reading coaches in mature Reading First schools reported that they spend 
much more of their time on coaching responsibilities than do reading coaches in Title I 
schools (86 percent vs. 59 percent). 



Because respondents from non-RF Title I schools who completed the Reading Coach Survey reported a 
multiplicity of job titles, we used their responses to two survey questions to determine their inclusion in the 
comparison group of reading coaches: How central is each of the following activities? 1) “Coaches staff on a 
range of topics”; and 2) “Organizes professional development for K-3 teachers.” Respondents who 
answered a 3 (“somewhat central”) or above (on a five-point scale) for at least one of these two items were 
included in the comparison group of reading coaches from Title I schools for these sets of analyses regarding 
reading coach responsibilities (Exhibits 6.5 and 6.7). As a result, 34 reading coach respondents from Title I 
schools, the equivalent of 940 weighted respondents, were excluded from these analyses. 
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Exhibit 6.1 

Responsibiiities of the Reading Coach in Mature Reading First and Titie i Schoois, 2004-05 



School Year 


Responsibilities of 


Reading First 


Title 1 


Reading Coach 


Schools 


Schools 


Average number of schools with which reading coach 
works 


1.2 


1.4* 


... 1 school 


88% 


83% 


... 2 schools 


9 


8 


... 3 schools 


1 


1 


... 4+ schools 


1 


6 


Average number of teachers with whom reading coach 
works 


22.0 


23.3 


... 1-10 teachers 


16% 


16% 


... 11-20 teachers 


38 


33 


... 21-30 teachers 


27 


28 


... 31 + teachers 


19 


21 


Average percentage of time spent as reading coach 


86%* 


59% 


... 100% time 


58 


19 


... 75-99% time 


20 


16 


... 50-74% time 


16 


37 


... 25-49% time 


3 


17 


... 1-24% time 


2 


11 


Source: Reading Coach Survey, Questions Bl- B3. 
Weight: Reading coach. 



Weighted respondents: 2,1 16 RF reading coaches, 6,727 reading coaches in Title I schools. 

Nonresponse rates across survey items: 0.7 to 3.9 percent. 

Exhibit reads: A reading coach in an RF school serves an average of 1.2 schools, compared with 1.4 schools for a reading 
coach in a Title I school. This difference is statistically significant (p < .05). 88 percent of reading coaches in RF schools 
reported serving just one school, compared with 83 percent of reading coaches in Title I schools. 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
schools. 



Responsibilities of Reading Coaches 

We asked reading coaches to rate the relative importance of a series of different activities that 
are central to the Reading First program; their responses provide an indication of the fidelity 
with which reading coaches are implementing Reading First in the total population of RF 
schools. 

Examining teacher support activities (Exhibit 6.2), more than 85 percent of the reading 
coaches in the total population of RE schools characterized three activities as absolutely 
central to their work: providing professional development (94 percent), coaching school staff 
(91 percent), and organizing professional development for K-3 teachers (87 percent). The task 
least likely to be rated as central to coaches’ work in the total population of Reading Eirst 
schools was providing direct reading instruction to students (rated as not central by 49 percent 
of reading coaches). On the other hand, there is variation across reading coaches in the 
perceived importance of teaching reading directly to students, as 30 percent of reading 
coaches in RE schools rated this task as absolutely central. Although there is no significant 
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difference between the percentage of reading coaches at new and mature RF schools rating 
this as absolutely central, a significant difference does exist between new and mature RF 
schools in the percentage of reading coaches rating direct instruction to students as not central 
to their role as reading coach (44 percent vs. 52 percent), indicating that there may be a shift 
in reading coaches’ role as Reading First becomes more firmly planted in their school. This 
is a potential cause for concern, because reading coaches in RF schools are supposed to be 
focusing on providing support and professional development to teachers, rather than on 
instructing students. However, in smaller schools, reading coaches might have the time to 
both serve teachers’ needs and provide direct reading instruction to students. To explore this 
issue further, reading coach response was analyzed by school size. Consistent with this 
hypothesis, 44 percent of reading coaches from schools with fewer than 250 students 
responded that direct instruction to students was absolutely central, the highest percentage of 
the four school size categories analyzed."^^ 



We will be able to test this hypothesis with longitudinal data after the next wave of data collection for the 
“new” RF schools in the 2006-07 school year. That is, by asking reading coaches from the same schools 
after two more years of RF implementation, we will be able to see if there is a change in these figures. 

Additional figures: For school size 250-499, 31 percent; 500-749, 21 percent; and greater than or equal to 
750, 29 percent of reading coaches rated “providing direct instruction to students” as absolutely central to 
their role as reading coach. 
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Exhibit 6.2 



Reading Coaches’ Ratings of the importance of Teacher Support Activities Reiated to the Reading 
Programs in Reading First Schoois, 2004-05 Schooi Year 

Teacher Support Activities 

Percent 



100% 

80% 

60% 

40% 

20 % 

0% 




Facilitate grade level 


Provide direct 


Provide training/ 


Coach staff on a 


Organize 


meetings 


reading instruction to 
students 


professional 
development in 
reading materials, 
strategies, and 
assessments 


range of topics 


professional 
development for K-3 
teachers 



■ ...Not central (1 or 2) ...Somewhat central (3) ■ ...Absolutely central (4 or 5) 



Source: Reading Coach Survey, Question B4. 

Weight: Reading coach 

Weighted respondents: 3,754 RF reading coaches. 

Nonresponse rates across survey items: 0.7 to 1.2 percent. 

Exhibit reads: 5 percent of reading coaches at RF schools reported administering and coordinating reading assessments as “not 
central” to their work; 8 percent reported it as “somewhat central”; and 87 percent reported it as “absolutely central.” 



Reading coaches in mature Reading First schools were significantly more likely than their 
Title I counterparts to report that the following reading activities were absolutely central to 
their work (Exhibit 6.3): providing training or professional development (95 percent vs. 87 
percent), coaching staff on a range of topics (92 percent vs. 83 percent), organizing 
professional development (87 percent vs. 67 percent), and facilitating grade-level meetings 
(67 percent vs. 47 percent). On the other hand, reading coaches in Title I schools were 
significantly more likely than Reading First reading coaches to report that provision of direct 
reading instruction to students is absolutely central to their work (63 percent vs. 29 percent). 
The role of the “reading coach” in many Title I schools thus seems to be more like the 
traditional “reading specialist,” with more emphasis on directly teaching students reading. 
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Exhibit 6.3 



Reading Coaches’ Ratings of the importance of Various Teacher Support Activities in RF 
and Titie i Schoois, 2004-05 Schooi Year 



Reading Activity/ 


New Reading 
First Schools 


Mature Reading 
First Schools 


Title 1 
Schools 


Centrality of the Activity 


Percent 


Percent 


Percent 


Facilitate grade-level meetings 
...Not central (1 or 2) 


11% 


14% 


30%* 


...Somewhat central (3) 


20 


19 


23 


...Absolutely central (4 or 5) 


69 


67* 


47 


Coach staff on a range of topics 
...Not central (1 or 2) 


3 


2 


3 


...Somewhat central (3) 


7* 


6 


14* 


...Absolutely central (4 or 5) 


91 


92* 


83 


Provide direct reading instruction to students 
...Not central (1 or 2) 


44 


52* 


28 


...Somewhat central (3) 


24 


19 


19 


...Absolutely central (4 or 5) 


32 


29 


53* 


Organize professional development for K-3 
teachers 

...Not central (1 or 2) 


5 


3 


15* 


...Somewhat central (3) 


7 


10 


18* 


...Absolutely central (4 or 5) 


88 


87* 


67 


Provide training/professional development in 
reading materials, strategies, and assessments 
...Not central (1 or 2) 


2 


2 


5* 


...Somewhat central (3) 


5 


3 


9* 


...Absolutely central (4 or 5) 


93 


95* 


87 



Source: Reading Coach Survey, Question B4. 

Weight: Reading coach. 

Weighted respondents: 1,630 new RF reading coaches, 2,124 mature Reading First reading coaches, 6,727 reading 
coaches in Title I schools. 

Nonresponse rates across survey items: 0.4 to 2.1 percent. 

Exhibit reads: 67 percent of reading coaches in mature RF schools rate facilitating grade-level meetings as “central” to 
their work, compared with 47 percent of reading coaches in Title I schools. This difference is statistically significant (p < 
.05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between mature RF and 
Title I schools. 



Turning to administrative support activities (Exhibit 6.4), more than 85 percent of the reading 
coaches in the total population of RF schools characterized two activities as absolutely central 
to their work: compilation of reading assessment data (88 percent), and administration and 
coordination of reading assessments (87 percent). Only about one-third of the reading coaches 
in the total population of RF schools reported facilitation of family literacy activities (34 
percent) or provision of sub time for teachers to observe experienced teachers (33 percent) as 
absolutely central to their work. This finding is not surprising, because these two activities are 
not part of the Reading First guidelines for the reading coach’s role. 

Nearly all of the reading coaches in the total population of RF schools characterized one 
activity that did not fall into the “teacher support” or “administrative support” categories 
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(Exhibit 6.4) as absolutely central to their role: participation in professional development (96 
percent). 



Exhibit 6.4 



Reading Coaches’ Ratings of the importance of Administrative and Schooi Support Activities 
Reiated to the Reading Programs in Reading First Schoois, 2004-05 Schooi Year 



Percent 
100% . 

80% 

60% 

40% 



20 % 



0% 



Administer/ 

coordinate 

reading 

assessments* 



Administrative and Schooi Support Activities 







Compile 

reading 

assessment 

data* 



Participate in Facilitate or Order/manage Participate in Provide sub 



school coordinate reading 

leadership family literacy instruction 
team meetings activities* materials* 



professional time for 
development teachers to 

provided by the observe 

district, state or experienced 
other teachers* 

consultants 



I ...Not central (1 or 2) ...Somewhat central (3) ■ ...Absolutely central (4 or 5) 



Source: Reading Coach Survey, Question B4. 

Weight: Reading coach. 

Weighted respondents: 3,754 RF reading coaches. 

Nonresponse rates across survey items: 0.3 to 1.7 percent. 

Exhibit reads: 5 percent of reading coaches at RF schools reported administering and coordinating reading assessments as “not 
central” to their work; 8 percent reported it as “somewhat central”; and 87 percent reported it as “absolutely central.” 

Note: Percents may not always add up to 100 due to rounding. 

The star (*) indicates an “administrative support activity.” 



Reading coaches in mature RF schools were significantly more likely than their Title I 
counterparts to report that the following administrative support activities were absolutely 
central to their work (Exhibit 6.5): compiling reading assessment data (88 percent vs. 67 
percent), and administering or coordinating reading assessments (86 percent vs. 67 percent). A 
majority of reading coaches in both RF and Title I schools reported that participating in school 
leadership team meetings and managing reading instruction materials were absolutely central 
to their work. In addition, significantly more reading coaches in mature RF schools 
characterized participating in professional development (97 percent vs. 89 percent) as 
absolutely central to their work. 
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Exhibit 6.5 



Reading Coaches’ Ratings of the importance of Administrative and Schooi Support 
Activities, in RF and Titie i Schoois, 2004-05 Schooi Year 



Reading Activity/ 


New Reading 
First 
Schools 


Mature 
Reading First 
Schools 


Title 1 
Schools 


Centrality of the Activity 


Percent 


Percent 


Percent 


Administer/coordinate reading assessments 
...Not central (1 or 2) 


4% 


6%* 


14% 


...Somewhat central (3) 


8 


8* 


19 


...Absolutely central (4 or 5) 


88 


86* 


67 


Compile reading assessment data 
...Not central (1 or 2) 


5 


4* 


14 


...Somewhat central (3) 


7 


8* 


19 


...Absolutely central (4 or 5) 


88 


88* 


67 


Facilitate or coordinate family literacy activities 
...Not central (1 or 2) 


38 


42 


39 


...Somewhat central (3) 


27 


24 


28 


...Absolutely central (4 or 5) 


34 


34 


33 


Order/manage reading instruction materials 
...Not central (1 or 2) 


11 


11 


15 


...Somewhat central (3) 


13 


20 


15 


...Absolutely central (4 or 5) 


76 


69 


70 


Provide sub time for teachers to observe other 
more experienced teachers* 

...Not central (1 or 2) 


40 


43 


56* 


...Somewhat central (3) 


24 


27 


18* 


...Absolutely central (4 or 5) 


36 


30 


26 


Participate in school leadership team meetings 
...Not central (1 or 2) 


9 


9 


12 


...Somewhat central (3) 


14 


15 


13 


...Absolutely central (4 or 5) 


78 


76 


75 


Participate in professional development provided 
by the district, state or other consultants 
...Not central (1 or 2) 


0 


1* 


4 


...Somewhat central (3) 


3 


3* 


7 


...Absolutely central (4 or 5) 


96 


97* 


89 



Source: Reading Coach Survey, Question B4. 

Weight: Reading coach. 

Weighted respondents: 1,630 reading coaches in new RF schools, 2,124 reading coaches in mature RF schools, 6,727 
reading coaches in Title I schools. 

Nonresponse rates across survey items: 0.3 to 1.6 percent. 

Exhibit reads: 86 percent of reading coaches in RF schools rate administering/coordinating reading assessments as 
“central” to their work, compared with 67 percent of reading coaches in Title I schools. This difference is statistically 
significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 
Title I schools. 



Coaching Activities Undertaken by Reading Coaches 

Reading coaches also were asked about the importance of different coaching activities to their 
work (Exhibit 6.6). In general, reading coaches in the total population of Reading First 
schools thought that all coaching activities were important, and rated the activities an average 
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of 4.2 on a five-point scale (1 = not central and 5 = absolutely central). Six of 10 coaching 
activities were perceived as absolutely central to at least 85 percent of the reading coaches; 
these activities included helping teachers to interpret assessment results (93 percent), to design 
strategies for struggling readers (91 percent), to use the core reading program (89 percent), 
and to assess the effectiveness of strategies for struggling readers (87 percent); these activities 
also included observing and providing feedback to teachers (85 percent), and assisting 
teachers to form instructional groups (84 percent). The coaching activity characterized as least 
central — though deemed central by approximately half (5 1 percent) of the coaches — was 
reviewing teachers’ lesson plans and providing feedback. 

Reading coaches in mature RF schools were significantly more likely than reading coaches in 
Title I schools to report that all of these coaching activities were absolutely central to their 
work (Exhibit 6.7), with two exceptions: helping teachers monitor the effectiveness of 
strategies for struggling readers (88 percent vs. 83 percent); and reviewing lesson plans and 
providing feedback (25 percent vs. 22 percent). 
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Exhibit 6.6 

Reading Coaches’ Ratings of the importance of Activities Supporting Teachers’ instruction in Reading First Schoois, 2004-05 Schooi 
Year 



Percent 




Give 


Assist 


Observe and 


Assist 


Help teachers 


Help teachers 


Give 


Plan reading 


Review 


Assist teachers 


demonstration 


teachers in 


provide 


teachers in 


design 


assess 


demonstrations 


instruction 


teachers’ 


in interpreting 


iessons using 


using the 


feedback to 


forming 


strategies for 


effectiveness 


on assessment 


with teachers 


lesson 


assessment 


core/ 


core 


teachers 


instructional 


struggling 


of strategies 


administration 




plans and 


results 


suppiemental 


program 




groups 


readers 


for struggling 


and scoring 




provide 




materials 










readers 






feedback 





■ . ..Not central (1 or2) ...Somewhat central (3) ■...Absolutely central (4 or 5) 



Mean score for 10 items 4.2 

Source: Reading Coach Survey, Question B5. 

Weight: Reading coach. 

Weighted respondents: 1 1,378 RF reading coaches. 

Nonresponse rates across survey items: 0.5 to 1.5 percent. 

Exhibit reads: 76 percent of reading coaches in RF schools rated giving demonstration lessons using core/supplemental materials as “absolutely central” to their work, and 7 
percent rated it as “not central.” 























Exhibit 6.7 



Reading Coaches’ Ratings of the importance of Activities Supporting Teachers’ 
instruction in Mature Reading First and Titie i Schoois, 2004-05 Schooi Year 



Coaching Activity/ 
Centrality of the Activity 


Reading First 
Schools 
Percent 


Title 1 
Schools 
Percent 


Give demonstration iessons with core/supplementai materiais 
...Not centrai (1 or 2) 


5% 


1 3%* 


...Somewhat centrai (3) 


16 


13 


...Absoiuteiy centrai (4 or 5) 


79 * 


70 


Assist teachers in using the core program 
...Not centrai (1 or 2) 


2 


9* 


...Somewhat centrai (3) 


9 


13 


...Absoiuteiy centrai (4 or 5) 


89 * 


77 


Observe and provide feedback to teachers 
...Not centrai (1 or 2) 


4 


17* 


...Somewhat centrai (3) 


12 


13 


...Absoiuteiy centrai (4 or 5) 


84 * 


69 


Assist teachers in forming instructionai groups 
...Not centrai (1 or 2) 


4 


12* 


...Somewhat centrai (3) 


11 


20* 


...Absoiuteiy centrai (4 or 5) 


85 * 


68 


Heip teachers design strategies for struggiing readers 
...Not centrai (1 or 2) 


3 


4 


...Somewhat centrai (3) 


6 


11 


...Absoiuteiy centrai (4 or 5) 


90* 


86 


Heip teachers monitor the effectiveness of strategies for 
struggiing readers 
...Not centrai (1 or 2) 


4 


7 


...Somewhat centrai (3) 


9 


11 


...Absoiuteiy centrai (4 or 5) 


88 


83 


Give demonstrations on assessment administration/scoring 
...Not centrai (1 or 2) 


13 


18* 


...Somewhat centrai (3) 


16 


26* 


...Absoiuteiy centrai (4 or 5) 


71 * 


56 


Pian reading instruction with teachers 
...Not centrai (1 or 2) 


9 


13 


...Somewhat centrai (3) 


17 


24 


...Absoiuteiy centrai (4 or 5) 


73 * 


64 


Review teachers’ iesson pians & provide feedback 
...Not centrai (1 or 2) 


54 


57 


...Somewhat centrai (3) 


21 


21 


...Absoiuteiy centrai (4 or 5) 


25 


22 


Heip teachers in interpreting assessment resuits 
...Not centrai (1 or 2) 


2 


10* 


...Somewhat centrai (3) 


3 


17* 


...Absoiuteiy centrai (4 or 5) 


94 * 


72 



Source: Reading Coach Survey, Question B5. 

Weight: Reading coach. 

Weighted respondents: 2,1 13 RF reading coaches, 6,701 reading coaches in Title I schools. 

Nonresponse rates across survey items: 0.5 to 1.2 percent. 

Exhibit reads: 79 percent of reading coaches at RF schools rated giving demonstration lessons with 
core/supplemental materials as “absolutely central” to their work, compared with 70 percent of reading coaches 
in Title I schools (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 
Title I schools. 



80 



Chapter 6: Oversight and Classroom Support Activities 

















Responsibility for Oversight of Reading Activities 



Principals in Reading First schools (new and mature) were asked to indicate what entity is 
responsible for supporting and providing oversight of classroom reading activities — the state, the 
school district, or the school. The responses show that school personnel, including the principal 
or reading coach, were almost always solely responsible or jointly responsible, along with 
district and state personnel, for all forms of oversight of reading activities. Only rarely were 
district or state personnel solely responsible for reading oversight. In about 90 percent of the total 
population of RF schools, school principals and reading coaches were either solely or jointly 
responsible for monitoring the implementation of the reading program, for reviewing teachers’ 
lesson plans, for reviewing student progress in reading, for interpreting assessment results, for 
providing feedback to teachers about reading instruction, and for selecting professional 
development topics and opportunities. 

In general, mature Reading First and non-RF Title I schools differ in the assignment of 
responsibility for reading activities oversight (Exhibit 6.8). School staff alone, or school staff in 
combination with district staff, were significantly more likely to be responsible for monitoring 
the implementation of reading programs in Title I schools than in mature RF schools, in which 
the responsibility was more likely to be shared among school, district, and state staff. 
Additionally, RF schools were significantly more likely to report shared responsibility among 
school, district, and state staff for interpreting assessment results, giving feedback to teachers 
about reading instruction, and selecting reading professional development topics and 
opportunities; Title I schools, however, were significantly more likely to report responsibility 
shared between just school and district personnel for those activities. For example, reading 
coaches in Title I schools were somewhat more likely to report having the support of their 
principals than were reading coaches in RF schools. The difference, while statistically 
significant, is small."^"* 

Furthermore, mature RF schools were significantly more likely to share responsibility among 
school, district, and state staff in reviewing teachers’ reading lesson plans and reviewing 
individual students’ progress in reading. This general trend of more state involvement in Reading 
First schools reflects the structure of the Reading First program, which requires states to be 
involved in designing and overseeing the Reading First program in grantee schools. 



Reading coaches were asked to indicate on a scale of one to five how accurately the statement “Reading 
coaches have the support of the principal” (Question HI)” describes their school. The average response was 
4.43 for RF schools and 4.60 for Title I schools. This difference is statistically significant (p < .05). 
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Exhibit 6.8 

Responsibiiity for Oversight of Reading Activities in Mature Reading First and Titie i 
Schoois, 2004-05 Schooi Year 


Reading Oversight Activity/ 
Responsible Party 


Reading First 
Schools 
Percent 


Title 1 
Schools 
Percent 


Monitoring impiementation of reading program 






...Sohooi aione is responsibie 


32% 


48%* 


...Schooi and district are jointiy responsibie 


28 


38* 


...Schooi, district and state are jointiy responsibie 


33 


8* 


...Other combinations 


7 


7 


Review of Teachers’ reading lesson plans 






...School alone is responsible 


91 


89 


...School and district are jointly responsible 


3 


6 


...School, district and state are jointly responsible 


2* 


0 


...Other combinations 


3 


5 


Review individual students’ progress in reading 






...School alone is responsible 


64 


65 


...School and district are jointly responsible 


18 


22 


...School, district and state are jointly responsible 


13* 


6 


...Other combinations 


5 


7 


Interpretation of assessment results 






...School alone is responsible 


40 


40 


...School and district are jointly responsible 


35 


42* 


...School, district and state are jointly responsible 


21* 


12 


...Other combinations 


5 


6 


Feedback to teachers about reading instruction 






...School alone is responsible 


72 


70 


...School and district are jointly responsible 


14 


23* 


...School, district and state are jointly responsible 


8 


4* 


...Other combinations 


6 


4 


Selection of reading professional development topics 






and opportunities 






...School alone is responsible 


27 


33 


...School and district are jointly responsible 


45 


52* 


...School, district and state are jointly responsible 


18* 


4 


...Other combinations 


10 


12 


Source: Principal Survey, Question D5. 

Weight: Principal. 

Weighted respondents: 3,866 RF principals, 14, 485 Title I principals. 

Nonresponse rates across survey items: 0.3 to 1.5 percent. 

Exhibit reads: 32 percent of RF schools reported that school personnel alone are responsible for monitoring the 
implementation of the reading program, compared to 48 percent of Title I schools. This difference is statistically 
significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
schools. 



82 



Chapter 6: Oversight and Classroom Support Activities 







Role of the School Principal in the Evaluation of K-3 Reading 
Instruction 



School principals were asked how they evaluated the K-3 reading instruction in their sehools. 
The most common form of evaluation in the total population of Reading First sehools was 
informal evaluation through observation, done at least weekly by 64 percent of the principals. 

There were few significant differences between mature Reading First and Title I schools in 
principals’ evaluations of K-3 reading instruction (Exhibit 6.9). Principals in mature Reading 
First schools were significantly more likely than Title I school principals to observe classroom 
reading instruction at least weekly (64 percent vs. 46 pereent); RF principals were also 
significantly more likely than Title I principals to meet monthly with groups of teachers to 
discuss strategies for improving reading instruction. 



Chapter 6: Oversight and Classroom Support Activities 



83 





Exhibit 6.9 

Percentage of Reading First Schoois and Titie i Schoois in Which K— 3 Reading instruction 
Was Evaiuated by the Principai, by Mode and Frequency of Evaiuation, 2004—05 Schooi Year 



Mode of Evaluation/ 


Reading First 
Schools 


Title 1 
Schools 


Frequency of Evaluation 


Percent 


Percent 


Observed classroom reading instruction informally 


...Not at all 


0% 


0% 


...1-4 times this year 


6 


15* 


...5-8 times this year 


14 


15 


...Once a month 


17 


23* 


. . .Once a week or more 


64* 


46 


Observed elassroom reading instruction using an 
evaluation form 


...Not at all 


4 


7* 


...1-4 times this year 


38 


44 


...5-8 times this year 


19 


15 


...Once a month 


22 


18 


. . .Once a week or more 


18 


15 


Met with teachers individually to discuss strategies for 
improving reading instruction 


...Not at all 


4 


5 


...1-4 times this year 


32 


35 


...5-8 times this year 


22 


21 


...Once a month 


27 


24 


. . .Once a week or more 


16 


15 


Met with groups of teachers to disouss strategies for 
improving reading instruction 


...Not at all 


2 


3 


...1-4 times this year 


18 


28* 


...5-8 times this year 


21 


24 


...Once a month 


40* 


30 


. . .Once a week or more 


18 


15 



Source: Principal Survey, Question D6. 

Weight: Principal. 

Weighted respondents: 3,866 RF principals, 14, 485 Title I principals. 

Nonresponse rates across survey items: < 1 percent. 

Exhibit reads: 14 percent of principals in RF schools reported observing classroom reading instruction informally 5- 8 
times this year, compared to 15 percent of principals in Title I schools. This difference is not statistically significant. 
Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
schools. 



Summary 

Overall, it appears that teachers in RF schools have more support for their reading instruction 
than teachers in non-RF Title I schools. One element of classroom support is access to a reading 
coach, whose main role is to help teachers develop their skills as reading instructors. As required 
by most states, nearly all RF schools reported having a reading coach, which was significantly 
more than was true for non-RF Title I schools (98 percent vs. 60 percent). Reading coaches in 
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mature Reading First schools were significantly more likely than their non-RF Title I 
counterparts to report that many teacher support and coaching activities recommended for 
reading coaches were absolutely central to their work. They were, correspondingly, less likely 
than reading coaches in non-RF Title I schools, to identify as central the provision of direct 
reading instruction to students (29 percent vs. 53 percent). At the same time, RF reading coaches 
were also significantly more likely than reading coaches in non-RF Title I schools, to rate several 
administrative activities as absolutely central to their work. 

The state was significantly more involved in oversight of school reading programs, including 
monitoring implementation (33 percent"^^ percent vs. 8 percent), interpretation of reading 
assessment results (21 percent vs. 12 percent), and selecting professional development topics in 
reading (18 percent vs. 4 percent), in RF schools than in non-RF Title I schools. However, 
school personnel (including the principal or reading coach) were also nearly always involved in 
oversight activities in both RF and non-RF Title I schools, and sometimes were solely 
responsible for oversight activities, such as reviewing teachers’ lesson plans in reading and 
providing feedback to teachers about reading instruction. In addition, nearly all states have 
developed guidelines for monitoring and evaluating the RF programs. 



Percents are for the category “state, district and school are jointly responsible.” It was extremely rare for the 
state to be solely responsible for any of these activities. 
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Chapter 7: Professional Development 

The RF program provides funds to be used at the state and local levels to (1) ensure that 
research-based reading programs and materials are used to teach students in K-3, (2) increase 
access and quality of professional development of all teachers who teach K-3 students, thereby 
ensuring that they have effective skills for teaching reading, and (3) help prepare classroom 
teachers to screen, identify, and overcome barriers to students’ ability to read on grade level by 
the end of third grade. Taken together, these goals indicate that Reading First aims to strengthen 
teacher knowledge about teaching reading. 

The RF legislation provides several strategies to meet this objective, one of which addresses 
teacher professional development focused on scientifically based reading practices. This focus on 
professional development reflects the view that reading research currently provides a strong, 
evidence-based understanding of how to teach reading effectively in the early grades, 
particularly to children who are struggling academically. The legislation specifies that states 
offer comprehensive professional development to help teachers work with these students, as well 
as to enrich the overall quality of reading instruction. It also calls for building teachers’ working 
knowledge of the reading research base at conceptual and operational levels (e.g., understanding 
conceptual principles as well as knowing how to sequence practices mostly effectively to impact 
children’s development of reading skills). 

Professional development provided through Reading First should reflect current research on 
reading instruction and the use of scientifically based reading instructional techniques. The 
underlying intention is that through participation in research-based professional development, 
teachers will better be able to meet the needs of all of their K-3 students, including struggling 
readers. Survey responses indicate that teachers in Reading First schools are being offered and 
are attending a wide range of professional development activities that cover myriad topics related 
to teaching reading. It is important to note that some professional development 
opportunities offered as part of Reading First are available to all K-3 teachers in the state 
regardless of whether or not their school is receiving RF funds. Therefore, the Title I 
survey responses could reflect staff participation in these opportunities. 

States have a central role in the development and provision of both statewide and district- level 
professional development for RF schools. RF coordinators in 33 states reported that the state 
chooses and organizes all statewide professional development (PD) efforts for Reading First. 
State staff also play a key role in selecting PD topics for local or district-based professional 
development for RF schools. 

States have implemented a variety of PD delivery systems for offering teachers professional 
development. The majority of states’ professional development plans include a “train-the- 
trainer” model where state-level trainers (other state agency staff, or a group contracted by the 
state) train representatives from districts and schools — such as reading coaches and a team of 
classroom teachers — who, in turn, train other teachers and reading-related staff in their districts 
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and schools. Other state plans included regional staff-led trainings for teams from each school. 
These regional systems have often been coupled with distance learning supports, such as 
listservs, to bolster regional training once teachers return to their classrooms, because direct 
contact with the school-based staff is limited. Still other states have forged strong relationships 
with their state university systems, resulting in professional development offerings that are based 
on this collaboration. Courses are often held at university campuses but designed by both faculty 
and state education agency personnel to meet SBRR (or other) content needs related to RF 
implementation. 

Many states have offered specialized training to district and school staff according to 
professional roles. For example, states first trained district reading specialists, then offered 
training to school-level reading coaches, and then trained all K-3 reading and classroom 
teachers. State directors, as part of their RF state plan, indicated that this approach “triaged” the 
most important individuals for the implementation of the Reading First plan so that district-level 
reading staff, for example, who have a larger role in RF implementation — including 
accountability for school-level implementation — are trained before classroom teachers receive 
training. 

This chapter summarizes findings related to five aspects of professional development including: 
a) the structure of activities (e.g., workshops, conferences); b) specific design features (e.g., 
availability of stipends, required attendance); c) activities related to the//ve dimensions of 
reading instruction-, d) activities related to teaching strategies for reading instruction, and e) 
direct classroom support to teachers for teaching reading (e.g., classroom demonstrations, 
coaching). 

The Structure of Professional Development Activities Attended by 
Teachers 

RF teachers reported participating in significantly more professional development activities than 
did Title I teachers. This is corroborated by the reading coach reports; significantly more RF 
coaches reported that sufficient time is allotted for professional development than did reading 
coaches in Title I schools."^^ Virtually all RF teachers (94 percent) attended either half- or full- 
day workshops compared to 81 percent of Title I teachers (see Exhibit 7.1). Further, RF 
teachers, on average, spent significantly more hours participating in professional development 
activities than did Title I teachers (40 hours vs. 24 hours). There were no significant differences 
between RF and Title I teachers in attendance at conferences or college courses. 



Reading coaches were asked to indicate on a scale from one to five how accurately the statement “sufficient 
time during the school day is allotted for professional development,” (Question HI) describes their school. The 
average response in RF schools was 3.17 and 2.80 in Title I schools. This difference is statistically significant. 
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Exhibit 7.1 

Structure of Professionai Deveiopment Activities Attended by Teachers in Mature 
Reading First and Titie i Schoois, 2004-05 Schooi Year 



Reading First Teachers 


Title 1 Teachers 


Percent 


Percent 



Half-day workshops 


73%* 


65% 


Full-day workshops 


76* 


53 


Any workshop 


94* 


81 


College courses 


17 


14 


Conferences 


29 


26 






Mean 


Mean 


Number of workshops 


5.5* 


3.4 


Number of total hours In 


39.5* 


24.1 



attendance across all activities 

Source: Teacher survey, Question Dl. 

Weight: Teacher. 

Weighted respondents: 63,530 RF teachers, 205,375 Title I teachers. 

Nonresponse rate across survey items: 3.2 percent. 

Exhibit reads: Reading First teachers reported attending more half-day professional development workshops 
than Title I teachers during the last school year (73 percent vs. 65 percent). This difference is statistically 
significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between 
teachers in RF and Title I schools. 

See Appendix C, Exhibit C.7.1 for grade-level statistics. 



The Specific Design Features of Professional Development 

Teachers, principals, and reading coaches in RF schools reported that most of the school-based 
teacher professional development opportunities were led by well-established facilitators, were 
held at a convenient location, and were required (Exhibit 7.2). Also, a substantial proportion of 
the activities provided teachers with some incentive to participate in the form of release time 
(45-60 percent), stipends (41-49 percent), or graduate credits (about 20 percent). It is likely that 
professional development activities with these characteristics result in meaningful learning and 
training opportunities for participants (Corcoran, 1995; Corcoran, et ah, 2003; Caret, et ah, 1999; 
Learning First Alliance, 2003). 



There were several differences among teachers, principals, and reading coaches in how they 
characterized the professional development activities. Teachers and reading coaches were much 
less likely than principals to report that the professional development a) provided teachers with 
options (32, 32, and 70 percent, respectively), b) provided follow-up activities (49, 57, and 71 
percent, respectively), or c) provided release time for teachers (44, 50, and 60 percent, 
respectively). Principals were also more likely than teachers or reading coaches to report that the 
principal attended the activities (70, 55, and 59 percent, respectively). 
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Exhibit 7.2 



Characteristics of the Professionai Deveiopment Activities Reading First Teachers Attended in 
the Last Year, as Reported by Teachers, Principais and Reading Coaches, 2004-05 Schooi 
Year 



Professional development activities that: 


RF Teachers 


RF Reading 
Coaches 


RF Principals 


Percent 


Percent 


Percent 


are led by trainers or facilitators who have a 
well-established reputation 


76% 


84% 


88% 


are held in a convenient location (e.g., at 
school) 


73 


82 


84 


require teachers to attend 


77 


78 


80 


use a team-based approach (i.e., joint training 
of people who work together) 


69 


73 


78 


were attended by the principal 


55 


59 


70 


provide follow-up activities 


49 


57 


71 


include release time for participating teachers 


44 


50 


60 


provide a stipend 


41 


44 


49 


provide teachers options among which to 
choose 


32 


32 


70 


offer graduate college credits 


23 


21 


21 



Source: Teacher Survey, Question D3; Reading Coach Survey, Question F3; Principal Survey, Question FI. 

Weight: Teacher, reading coach and principal. 

Weighted respondents: 63,300 RF teachers, 3,724 RF reading coaches and 3,863 RF principals. 

Nonresponse rate across survey items and respondent types: 0.9 to 6.5 percent. 

Exhibit reads: Reading First reading coaches reported that 84 percent of the professional development activities offered to 
teachers were given by trainers or facilitators who have a well-established reputation. 



There were statistically significant differences between RF and Title I teachers’ reports of the 
features of professional development they received (Exhibit 7.3). In general, these findings are 
corroborated by the responses of reading coaches and principals. The professional 
development received by RF teachers, as compared with Title I teachers, was significantly 
more likely to offer incentives to participants in the form of stipends (40 percent vs. 20 
percent), release time (43 percent vs. 33 percent), or graduate credits (25 percent vs. 14 
percent). Additionally, there were three other features of the professional development received 
by RF teachers that may improve the overall quality and success of the activity. First, the 
activity was more likely to be conducted by a trainer with a well-established reputation (75 
percent vs. 65 percent). Second, professional development offerings were more likely to be 
held in a convenient location (73 percent vs. 62 percent). Third, the professional 
development was more likely to have provided follow-up activities (48 percent vs. 35 
percent), which is a central theme of the Reading First Program (Guidance for the Reading 
First Program, 2002, page 26) and improves the chances that the activity will positively 
affect reading instruction. 
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It is also worth noting that teachers reported that the principal attended only about half of the 
professional development activities attended by teachers. As the principal is the instructional 
leader in the school, we might have expected this percentage to be higher in RF programs. 



Exhibit 7.3 



Specific Features of Professionai Deveiopment Activities Differences Between Teachers in 
Mature Reading First and Titie i Schoois, 2004-05 Schooi Year. 




100 80 60 40 20 0 0 20 40 60 80 100 



Percent Percent 



Source: Teacher Survey, Question D3. 

Weight: Teacher. 

Weighted respondents: 37,130 RF teachers, 196,522 Title I teachers. 

Nonresponse rate across survey items: 3.5 to 7.3 percent. 

Exhibit Reads: Reading First teachers reported that 75 percent of the professional development activities offered to 
teachers were given by trainers or facilitators who have a well-established reputation, compared with 65 percent of the 
activities attended by Title I teachers. This difference is statistically significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between teachers in RF 
and Title I schools. 

See Appendix C, Exhibit C.7.3 for additional statistics. 
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Professional Development Related to the Five Dimensions of Reading 



At the core of Reading First are the five dimensions of reading instruction that have been 
identified in scientifically based reading research: phonemic awareness, decoding, vocabulary 
development, fluency, and comprehension. Mastery of these reading skills is critical to becoming 
a successful reader. By definition, reading programs in Reading First schools must include these 
five dimensions of effective reading instruction, integrated into a coherent instructional design. 
This means that teachers should be trained explicitly in these five dimensions as well as in 
teaching strategies for adapting these practices to the varying needs and circumstances of their 
students. Successful implementation of the selected reading programs depends on well-designed 
professional development activities that address these topics. 

Teachers’ Participation in Professionai Deveiopment on the Five Dimensions of 
Reading 

Exhibit 7.4 (right panel) presents the grade-level mean percentage of teachers who received 
professional development in each of the reading dimensions for RF and Title I teachers. 

Across all grades, RF teachers were significantly more likely than their Title I school 
counterparts to have participated in professional development activities focused on at least 
one of the five dimensions of reading instruction. 

• More than 90 percent of RF first grade teachers received professional development in 
phonemic awareness and decoding, compared with about 70 percent of Title I 
teachers. The same pattern holds even for the third grade, a grade in which more than 
three-fourths of RF teachers received professional development in these topics, 
compared with about 50 percent of Title I teachers. 

• With regard to vocabulary development, the grade-level means range from 67 to 78 
percent for RF teachers, compared with a range of 49 to 56 percent for Title I 
teachers. 

• Similarly, more than 85 percent of RF teachers at each of the grades received 
professional development in reading comprehension, compared with about 75 percent 
of Title I teachers at each grade. 



Five composite variables were constructed to summarize teacher responses to these subskill items; each 
composite represents teacher responses aggregated to one of the five “dimension” levels. If a teacher indicated 
participation in at least one professional development activity within a particular dimension, that teacher’ s 
composite value for that particular dimension was computed as “1”. 
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Exhibit 7.4 



Teacher Participation in Professionai Deveiopment Activities Reiated to the Five Dimensions of 
Reading: Teachers in Mature Reading First Schoois and Titie i Schoois, The Range of Grade-Levei 
Percents, 2004-05 Schooi Year 




Source: Teacher Survey, Question D4. 

Weight: Teacher. 

Weighted respondents: 38,249 RF teachers, 201,290 Title I teachers. 

Nonresponse rates across respondents: 1.9 percent. For the analysis of PD needs, an additional 16 percent of the responses were 
excluded because the respondents provided more than five responses, the limit specified in the survey directions. 

Exhibit reads: The grade-level means for RF teachers showing their participation in professional development in phonemic awareness 
range from 75 to 92 percent in grades K-3. This compares with a range of 50 to 74 percent for Title I teachers. At each grade, RF 
teachers received significantly more professional development on this topic than did Title I teachers. Notes: Each bar represents the 
range of the grade-level means computed for each item. 

Eor each grade level, a significant difference (p < .05) between RE teachers and Title I teachers is indicated by the grade level symbol 
(i.e., K, 1, 2, 3). For example, if kindergarten, second grade, and third grade showed significant differences between RF teachers and 
Title I teachers, then ”K23” would be presented. 

The fluency analysis is limited to teachers in first, second, and third grades. 

See Appendix C, Exhibit C.7.4 for the grade-level means and additional statistics. 



There were no statistically significant differences between RF and Title I teachers in their 
perceived needs for additional professional development in decoding, fluency building and 
comprehension (Exhibit 7.4, left panel). However, there were differences in teachers’ 
perceived need for additional professional development in phonemic awareness instruction; 
kindergarten and third-grade Title I teachers were more likely than their RF counterparts 
to indicate a need for additional training in these dimensions (kindergarten: 62 percent vs. 52 
percent; third grade: 35 percent vs. 24 percent). Additionally, about one-half of RF kindergarten 
teachers indicated a need for more training in teaching vocabulary, compared with 41 percent of 
Title I kindergarten teachers. 
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Teacher Ratings of Their Preparedness to Teach the Five Dimensions of Reading 

The objective of professional activities related to the dimensions of reading is to strengthen 
teachers’ knowledge of these component skills, thereby improving their teaching of reading. The 
majority of RF teachers, whether in newly funded or mature schools, feel well prepared to 
provide instruction in the five dimensions of reading. About three-fourths of the teachers feel 
well prepared to teach phonemic awareness"^^ and about two-thirds feel well prepared to teach the 
other dimensions. This means there are substantial numbers of teachers who do not rate 
themselves as well prepared to provide this type of instruction. This is not surprising in 
newly funded schools that are in the early stages of implementing their professional development 
plan. Of the teachers who rated themselves as well prepared to teach a specific reading 
dimension, almost all (85-95 percent) have received professional development on the specific 
dimension. Conversely, the teachers who rated themselves as not well prepared, were less likely 
to have reported participating in professional development on that topic (61 versus 79 percent). 

Across all five dimensions of reading, RF teachers across grades, on average, rated 
themselves as significantly better prepared to teach than did Title I teachers (Exhibit 7.5). 
Mean scores ranged from 3.83 on fluency to 4.13 on phonemic awareness for RF teachers, 
compared with a range of 3.40 to 3.66 for Title I teachers. This corresponds to results 
described earlier — that RF teachers reported receiving more professional development than did 
Title I teachers, across all of these components of reading instruction. 



Exhibit 7.5 



Preparedness to Teach Five Dimensions of Reading: Teachers in Mature Reading First and 
Titie i Schoois, 2004-05 Schooi Year 



Dimension 


RF Teachers 


Title 1 Teachers 


Mean 


Mean 


Phonemic Awareness 


4.13* 


3.66 


Decoding 


3.86* 


3.35 


Vocabuiary 


3.79* 


3.40 


Comprehension 


3.80* 


3.54 


Fiuency 


3.83* 


3.40 



Source: Teacher Survey, Question D6. 

Weight: Teacher. 

Weighted respondents: 37,683 RF teachers, 189,901 Title I teachers. 

Nonresponse rates across survey items: 6.1 to 6.6 percent. 

Exhibit reads: RF teachers rated themselves as better prepared (on a five-point scale) to teach phonemic awareness than 
did Title I teachers (4.13 vs. 3.66, p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between teachers in RF 
and Title I schools. 

See Appendix C, Exhibit C.7.5 for grade-level statistics. 



The analysis focused on phonemic awareness includes kindergarten and first-grade teachers only, because it is 
at these grades where the skill is heavily emphasized 
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Principals’ and Reading Coaches’ Participation in Professional Development on 
the Five Dimensions of Reading 

RF principals and reading coaches were significantly more likely to have participated in 
professional development than their Title I counterparts across all five dimensions of 
reading, with a 15-percentage point difference for principals, and 30-percentage point 
difference for reading coaches (Exhibit 7.6). These findings provide further evidence that the 
implementation of Reading First is resulting in a strong emphasis, at the school level, on the 
importance of providing reading programs that help students master these five key dimensions of 
reading. 



Exhibit 7.6 



Professionai Deveiopment Reiated to the Five Dimensions of Reading Attended by 
Principais and Reading Coaches in Mature Reading First and Titie i Schoois, 2004-05 
Schooi Year 



Dimension 


Reading First Schoois 


Titie i Schools 


Percent 


Percent 


Principai Participation 


Phonemic Awareness 


79%* 


63% 


Decoding 


72* 


58 


Vocabuiary 


80* 


66 


Comprehension 


84* 


67 


Fiuency 


83* 


75 


Reading Coach Participation 


Essentiai Components of Reading 


90* 


60 



Source: Principal Survey, Question F2a-e; Reading Coach Survey, Question Gif. 

Weight: Principal, reading coach. 

Weighted respondents: 2,182 RF principals, 14,259 Title I principals and 1 1,428 reading coaches. 

Nonresponse rates across survey items and respondents: 0.0 to 4.8 percent. 

Exhibit reads: 79 percent of principals in RF schools participated in professional development in phonemic 
awareness, compared with 63 percent of principals in Title I schools. The difference is statistically significant (p < 
.05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between teachers in 
RF and Title I schools. 



Professional Development Related to General Teaching Strategies 

The Reading First legislation specifies that professional development activities should also 
provide teachers with “information on instructional materials, programs and approaches based on 
scientifically based reading research” {Section 1202, D}. This information will (hypothetically) 
help teachers implement sound, research-based instructional practices that are most likely to help 
students develop mastery of the five dimensions of reading. 

Teachers’ Participation in Professional Development on General Teaching 
Strategies 

Teachers in Reading First schools were asked whether they had participated in professional 
development focused on 1) using materials and teaching strategies, 2) grouping, 3) assessment of 
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students, 4) how to help struggling readers, and 5) organization and planning. They were also 
asked to report in which of these topics they felt they needed additional professional 
development. Results are presented for these five categories of professional development topics 
in Exhibits 7.7a-d. 



Teachers in RE schools reported receiving professional development in a variety of topics related 
to materials and teaching strategies (Exhibit 7.7a). Most RE teachers (82 percent) reported 
receiving professional development in how to use their core reading programs, and the majority 
(60 percent) have received professional development in how to use reading research to guide 
instruction; indeed, this is one of the central aims of the Reading Eirst legislation. Only 1 1 
percent of RE teachers reported the need for additional professional development in this area. 

It is also noteworthy that substantial numbers of RE teachers received professional development 
in using children’s literature (53 percent) and integrating reading and writing instruction (59 
percent). In addition, about 20 percent of RE teachers perceived a need for additional 
professional development in using children’s literature and 30 percent indicated a need for more 
professional development in integrating reading and writing instruction. These teaching 
strategies, however, are not considered to be salient aspects of an effective Reading Eirst 
program. 



Exhibit 7.7a 



Teacher Professionai Deveiopment on Materiais and Teaching Strategies in Reading First Schoois, 
2004-05 Schooi Year 




How to use 
the core 
reading 
program 



How to use How to use How the core How to use How to Strategies for 
chiidren's reading reading the integrate teaching 

iiterature to research to program supplementai reading and reading to 

teach guide content incorporates reading writing ELLs 

chiidren of instruction research program(s) instruction 

principies 



Source: Teacher Survey, Question D5. 

Weight: Teacher. 

Weighted respondents: 59,728 RF teachers. 

Nonresponse rate: 8.3 percent. For the analysis of PD needs, an additional 26 percent of the responses were excluded because the 
respondents provided more than five responses, the limit specified in the survey directions. 

Exhibit reads: 84 percent of RF teachers reported receiving professional development in how to use the core reading program; 7 
percent reported needing additional professional development on this topic. 

Notes; If teachers provided more than five topics in which they need PD, their responses were excluded from the analysis. 

See Appendix C, Exhibit C.7.7a for grade-level statistics on these items. 
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Almost three-quarters of RF teachers reported receiving professional development on 
administering assessments and using the information to guide instruction (Exhibit 7.7b); this is a 
central tenet of Reading First, as noted in the Guidance for the Reading First Program: that “a 
high-quality reading program also includes assessment strategies for diagnosing student needs 
and measuring student progress as well as a professional development plan that ensures teachers 
have the skills necessary to implement the program effectively and meet the reading needs of 
individual students” (2002, page 6). Further, very few RF teachers reported needing additional 
training in these topics. It is important to note, however, that only about one-half of RF teachers 
(45 percent) reported receiving professional development in how to diagnose reading problems. 
Yet this is the topic most often perceived by RF teachers (38 percent) as one in which they need 
additional professional development. 

Almost half (45 percent) of RF teachers reported receiving professional development in learning 
styles, and almost a quarter (22 percent) felt they needed more professional development in this 
area. This again, is a topic that is not central to the Reading First program. 



Exhibit 7.7b 

Teacher Professionai Deveiopment on Grouping and Assessment in Reading First Schoois, 
2004-05 Schooi Year 




Learning styles How to organize How to diagnose How to How to interpret 
small group reading administer and use 

instruction problems assessments assessment data 

to guide 
instruction 

Grouping 1 I Assessment ' 



Source: Teacher Survey, Question D5. 

Weight: Teacher. 

Weighted respondents: 59,728 RF teachers. 

Nonresponse rate: 8.3 percent. For the analysis of PD needs, an additional 26 percent of the responses were excluded 
because the respondents provided more than five responses, the limit specified in the survey directions. 

Exhibit reads: 48 percent of RF teachers reported receiving professional development in learning styles; 22 percent 
reported needing additional professional development on this topic. 

Note: See Appendix C, Exhibit C.7.7b, for grade-level statistics on these items. 
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More RF teachers reported needing additional professional development in topics related to 
helping struggling readers master specific instructional content (see Exhibit 7.7c) rather than the 
broader topic of how their reading programs incorporate research principles (Exhibit 7.7a). 
Teachers reported wanting more training in how to help struggling readers with decoding (25 
percent), comprehension (30 percent), and motivation (30 percent). 



Exhibit 7.7c 



Teacher Professionai Deveiopment on Struggiing Readers in Reading First Schoois, 2004-05 
Schooi Year 




How to help 
struggling readers 
with decoding 



How to heip 
struggiing readers 
with vocabuiary 



How to heip 
struggiing readers 
with 

comprehension 



How to motivate 
readers 



Strategies for 
teaching reading 
to students with 
diagnosed 
learning 
disabilities 



Source: Teacher Survey, Question D5. 

Weight: Teacher. 

Weighted respondents: 59,728 RF teachers. 

Nonresponse rate: 8.3 percent. For the analysis of PD needs, an additional 26 percent of the responses were excluded because 
the respondents provided more than five responses, the limit specified in the survey directions. 

Exhibit reads: 64 percent of RF teachers reported receiving professional development in how to help struggling readers, with 
decoding; 25 percent reported needing additional professional development on this topic. 

Note: See Appendix C, Exhibit C.l.lc for grade level statistics on these items. 



Approximately two-thirds of RE teachers reported receiving professional development on how to 
use content standards for curriculum planning and teaching and how to align their curriculum 
and instruction with assessments. This indicates that many RE teachers are being given guidance 
in how to use content standards and assessments to inform their instruction. 



In general, only small proportions of RE teachers perceived the need for additional professional 
development in areas related to organization and planning (Exhibit 7.7d). Eor instance, under 10 
percent of teachers wanted more training in how to use state or district content standards (7 
percent) or alignment of instruction with assessments (8 percent); perhaps this reflects the fact 
that teachers feel they have little influence in these areas. Again, teachers were more likely to 
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report needing additional training on topics that focus on specific techniques and strategies, such 
as how to work with parents, and classroom management (22 and 16 percent). 



Exhibit 7.7d 



Teacher Professionai Deveiopment on Organization and Pianning in Reading First Schoois, 
2004-05 Schooi Year 




How to use 


How to align reading 


How to work with 


Ciassroom 


state/district content 


curricuium and 


parents 


management 


standards for 


instruction with 






curriculum planning 


state/district 






and teaching 


assessments 







Source: Teacher Survey, Question D5. 

Weight: Teacher. 

Weighted respondents: 59,728 RF teachers. 

Nonresponse rate: 8.3 percent. For the analysis of PD needs, an additional 26 percent of the responses were excluded 
because the respondents provided more than five responses, the limit specified in the survey directions. 

Exhibit reads: 69 percent of RF teachers reported receiving professional development in how to use state or district content 
standards for curriculum planning and teaching, 7 percent reported needing additional professional development on this topic. 
Note: See Appendix C, Exhibit C.7.7d for grade-level statistics on these items. 



In order to summarize teachers’ reported professional development, we constructed two types of 
scores. First, we calculated the proportion of teachers who reported receiving professional 
development on any of the topics in a given category. Second, we calculated the average number 
of topics in which teachers reported receiving professional development. We then used these 
summary measures to compare the professional development activities of RF and Title I teachers 
(Exhibit 7.8). 

Across all professional development topic areas related to reading instruction, RF teachers 
were significantly more likely than Title I teachers to have reported receiving professional 
development on at least one topic in each of the five categories. For example, 92 percent of 
RF teachers received professional development about at least one type of teaching strategy 
in the last year compared with 75 percent of Title I teachers. On average, RF teachers 
received professional development in four, and Title I teachers in three areas within this topic. 
Generally, this pattern of differences holds across the other areas of professional development. 
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Large proportions of RF and Title I teachers indicated the need for more training to work 
with struggling readers. In fact, significantly more Title I than RF teachers indicated such 
a need (76 percent vs. 60 percent). Many teachers, both RF and Title I who indicated a need for 
additional training on this topic had already received some training this year in working with 
struggling readers (76 and 60 percent). 

There were no meaningful (or statistically significant) differences between RF and Title I 
teachers in terms of their perceived professional development needs in the other broad topic 
areas: teaching strategies, grouping, assessment, and planning. 



Exhibit 7.8 






Professionai Deveiopment in Teaching Strategies in Mature Reading First and Titie i 
Schoois, 2004-05 Schooi Year 


Teachers 


Reading First 


Title 1 




Percent 


Percent 


Teaching strategies 


95%* 


83% 


Grouping 


73* 


62 


Assessment 


85* 


67 


Struggiing readers 


78* 


64 


Organization/ pianning 


80* 


75 


Number of Professional Development 
Activities 


Mean 


Mean 


Teaching strategies 


4.12* 


3.10 


Grouping 


1.12* 


0.95 


Assessment 


1.96* 


1.48 


Struggiing readers 


2.53* 


1.94 


Organization/ pianning 


2.20* 


1.96 


Source: Teacher Survey, Question D5. 

Weight: Teacher. 

Weighted respondents: 38,187 RF teachers, 196,060 Title I teachers. 

Nonresponse rates across survey items: 1.1 to 3.7 percent 

Exhibit reads: 92 percent of RF teachers compared to 75 percent of Title I teachers, received professional development 
in at least one instructional strategy. RF teachers attended a mean of 4.12 professional development activities in 
teaching strategies, compared to 3.10 for Title 1 teachers. This difference is statistically significant (p < .05). 

Notes: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between teachers in 
RF and Title I schools. 

See Appendix C, Exhibit C.7.8 for additional statistics. 



Principals’ Participation in Professional Development on General Teaching 
Strategies 

Principals were asked about the professional development they received in the previous year as 
well as their perceived needs for additional professional development. Exhibit 7.9 summarizes 
RF principals’ participation in professional development activities across several topic areas. 
Most principals participated in professional development activities related to teaching strategies 
(92 percent) and assessment (90 percent). Principals were more likely to participate in activities 
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related to teaching strategies than the other topic areas; on average their experiences covered 
about four different topics related to teaching strategies. On average, RF principals reported 
participating in 8.24 of 10 topic areas."^^ 



Exhibit 7.9 

Reading First Principais’ Participation in Professionai Deveiopment in 
Teaching Strategies, 2004-05 Schooi Year 


Topics 


RF Principals 
Percent 


Teaching strategies 


91% 


Evaiuation 


77 


Assessment 


90 


Struggiing readers 


70 


Organizaticn/ pianning 


76 


Composite Measure 


Mean 


Diversity cf PD Topics 


8.24 


Number of Professionai Deveiopment 
Activities 


Mean 


Teaching strategies 


4.00 


Grouping 


2.00 


Assessment 


2.32 


Struggiing readers 


2.57 


Organization/ pianning 


2.39 


Source: Principal Survey, Question F2. 

Weight: Principal. 

Weighted respondents: 3,829 RF principals. 

Nonresponse rate: 2.0 percent. 

Exhibit reads: 92 percent of RF principals participated in professional development on teaching 


strategies. 

Note: See Appendix C, Exhibit C.7.9 for principal participation rates 


on specific activities within 


each topic area. 





RF principals participated in a wider variety of professional development activities related 
to teaching strategies and the five dimensions of reading instruction than did principals in 
Title I schools (Exhibit 7.10). RF principals attended a significantly greater variety of 
professional development activities related to general teaching strategies (e.g., how to use 
the core reading program, how to use research to guide instruction) than did Title I 
principals (3.75 vs. 3.24). This same pattern holds for professional development related to 
assessment, evaluation, and working with struggling readers. There were no statistically 
significant differences between RF and Title I principals in the variety of professional 
development activities focused on organization, management, and support. 



To measure the diversity of professional development activities and topics for principals, we constructed a 
composite measure. This composite measure has a scale of 0-10, and it incorporates two kinds of professional 
development; those related to the five dimensions (each represented by one point), and those related to the five 
topic areas discussed above 1) teaching strategies, 2) evaluation, 3) assessment, 4) struggling readers, and 5) 
organization and planning. 
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These findings suggest that principals in RF schools have been provided with more 
comprehensive and targeted support related to comprehensive reading instruction. 



Exhibit 7.10 

Professionai Deveiopment Events Attended by Principais in Mature Reading First and Titie i 
Schoois: Summary and Composite Measures, 2004-05 Schooi Year 



Number of Events Attended by Principals 





Reading First Schools 


Title 1 Schools 


General Topics 


Mean 


Mean 


Teaching strategies 


3.75* 


3.24 


Assessment 


2.17* 


1.96 


Struggiing readers 


2.41* 


1.98 


Organization, management, and 
support 


2.26 


2.31 


Evaiuation 


1.85* 


1.42 



Source: Principal Survey, Question F2. 

Weight: Principal. 

Weighted respondents: 2,167 RF principals, 13,783 Title I principals. 

Nonresponse rates across respondents: 2.0 to 6.1 percent. 

Exhibit reads: Principals in RF schools reported attending, on average, 3.75 professional development activities about 
teaching strategies, compared with 3.24 for Title I principals. The difference in means is statistically significant (p < .05). 
Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 
Title I schools. 



Reading Coaches’ Participation in Professionai Deveiopment on Generai 
Teaching Strategies 

Reading coaches provide ongoing professional development to teachers both in modeling 
effective instructional practice and in providing overall support to teachers’ efforts to implement 
their reading programs. Indeed, the guidance for the Reading First Program states that, “delivery 
mechanisms [for professional development] should include the use of coaches and other teachers 
of reading who provide feedback as instructional strategies are put into practice” (2002, page 
26). Therefore, it is critical that coaches receive the support and training in how to effectively 
guide teachers and provide them with feedback regarding their teaching. 

Across all fifteen topics (Exhibit 7.11), reading coaches in RF schools were significantly 
more likely to have received professional development than reading coaches in Title I 
schools (differences ranging from 13 percentage points to 30 percentage points). The scores 
of RF coaches, on average, are higher than those of reading coaches in Title I schools. RF 
reading coaches were considerably more likely to have received professional development on 
how to help teachers make reading instruction systematic and explicit than reading coaches in 
Title I schools (78 percent vs. 52 percent). The same pattern holds for professional development 
in using assessment data to form instructional groups (90 percent vs. 68 percent), and how to 
plan instruction for struggling students (75 percent vs. 57 percent), and helping teachers with the 
essential components of scientifically based reading instruction (90 percent vs. 60 percent). In 
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summary, reading coaches in RF schools appear to be better prepared to support the teachers in 
their schools in implementing their reading instruction than coaches in Title I schools. 

Exhibit 7.11 

Topics Addressed in Professionai Deveiopment Offered to Reading Coaches in Mature 
Reading First and Titie i Schoois, 2004-05 Schooi Year 

Reading Coaches 



Reading First Title I 

Percent Percent 

1 . How to use reading assessment data to guide instruetion 94%* 79% 

2. What are the types of assessments: screening, diagnostic, 85* 55 

progress monitoring, and outcome 

3. How to use assessment data to form instructionai groups 90* 68 

4. How to provide constructive feedback to teachers 86* 65 

5. How to estabiish credibiiity with teachers 76* 45 

6. Essentiai components of scientificaiiy based reading instruction 90* 62 

7. What is the roie of the reading coach in fostering change 84* 55 

8. How to plan instructional interventions for struggling students 75* 58 

9. Classroom management within the literacy block time 57* 47 

10. How to conduct effective grade level meetings 60* 36 

1 1 . How to help teachers identify appropriate instructional materials 61* 45 

12. How to help teachers make reading instruction systematic and 78* 57 

explicit 

13. How to conduct demonstration lessons 68* 55 

14. How to conduct classroom observations 79* 54 

1 5. How to provide onsite professional development 66^ 55 

Source: Reading Coach survey. Question Gl. 

Weight: Reading coach. 

Weighted respondents: 2,025 RF reading coaches, 6,727 reading coaches in Title I schools. 

Nonresponse rate across survey items: 5 percent. 



Exhibit reads: 94 percent of Reading First reading coaches reported that professional development in using reading 
assessment data to guide instruction was offered to reading coaches in the last year, compared to 79 percent of reading 
coaches in Title I schools. The difference is statistically significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and 

Title I schools. 

Direct Classroom Support to Teachers for Improving Reading 
Instruction 

Teachers are often provided ongoing, direct support and feedback for their classroom teaching, 
above and beyond formal professional development. This includes attendance at grade-level 
meetings, help using assessment results to plan instruction, classroom demonstrations, 
observation of other teachers and peer study groups, and availability of support from reading 
coaches to improve instruction, among others. Teachers were asked to report whether or not they 
had received any of these types of support during the previous school year. 
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Across several types of direct support, a greater proportion of RF teachers reported 
receiving assistance with their reading instruction than did Title I teachers (Exhibit 7.12). 
Support for using assessment to guide instruction is significantly stronger in RF schools. 

• Eighty-nine percent of RF teachers received help in interpreting assessment data, 
compared with only 69 percent of Title I teachers. 

• Two-thirds of RF teachers indicated that they received help in using diagnostic tests with 
their students, compared with less than half of Title I teachers (66 percent vs. 49 percent). 

• Almost all RF teachers (91 percent) reported receiving support for using assessment data 
to determine topics that require additional instruction; this compares with about three- 
quarters of the Title I teachers (76 percent). 

These findings align with a critical element of Reading First — using students’ performance on 
reading tests to identify students at risk of failure and to modify reading instruction accordingly. 

Exhibit 7.12 also displays information about the availability of the various types of direct support 
for teachers’ reading programs, many of which are simply not available in Title I schools. For 
example, one-third of the teachers in Title I schools indicate that both coaching by a reading 
coach and diagnostic testing assistance is not available. 
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Exhibit 7.12 



Direct Support for Reading instruction for Teachers in Mature Reading First and Titie i 
Schoois, 2004-05 Schooi Year 





Teachers 




Support Received 


Support Not Available at My 
School 




Reading 

First 


Title 1 


Reading 

First 


Title 1 


Type of Support 


Percent 


Percent 


Percent 


Percent 


Coaching by reading coach in programs, 
materiais or strategies 


86%* 


50% 


3% 


32%* 


Coaching from feliow teacher 


58* 


47 


18 


26* 


Peer study group for group study 


47* 


32 


41 


55* 


Demonstrations in my ciassroom 


51* 


29 


15 


41* 


Observations of other teachers 


38* 


32 


24 


34* 


Diagnostic testing heip from a reading coach 
or speciaiist for individual students 


66* 


49 


12 


30* 


Intervention service help from a reading 
coach or specialist for individual students 


60* 


52 


14 


27* 


Interpretation of assessment data 


89* 


68 


3 


16* 


Grade level meetings devoted to reading 


91* 


76 


6 


17* 


Using assessment data to determine topics 
that require additional instruction 


91* 


76 


3 


16* 



Source: Teacher Survey, Question D2. 

Weight: Teacher. 

Weighted respondents: 38,217 RF teachers, 201,724 Title I teachers. 

Nonresponse rates across survey items: 1.0 to 3.0 percent. 

Exhibit reads: 86 percent of RF teachers have received assistance from a reading coach; this compares with 50 percent for 
Title I teachers. This difference is statistically significant (p < .05). 

Note: The stars (*) indicate that there is a statistically significant difference (i.e., p-value < .05) between RF and Title I 
schools. 



Many states require RF schools to have reading coaches on staff to provide mentoring and 
guidance to K-3 teachers as they implement their reading instruction. Indeed, RF coaches 
reported that teachers were given more time to be coached about their reading instruction from a 
reading coach^° and agreed more strongly that teachers sought their help to improve reading 
instruction than coaches in Title I schools. Most K-3 teachers (86 percent) reported that they 



Reading coaches indicated on a scale from “not at all” to “once a week or more” how much time was formally 
set aside for teachers to be coached about reading instruction by a reading coach. The average response for RF 
coaches was 4.01, corresponding to “once a month” whereas the average response for reading coaches in Title I 
schools was 2.35, corresponding most closely to “1-4 times per year.” This difference is statistically 
significant. 

Reading coaches were asked to indicate on a scale from one to five how accurately the statement “K— 3 teachers 
seek the assistance of the reading coach to improve their reading instruction,” (Question HI) describes their 
school. The average response in RF schools was 3.93 and 3.76 in Title I schools. This difference is statistically 
significant. 
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have received help from a reading coach about teaching strategies and instructional materials. 
About two-thirds of teachers indicated having received assistance in completing diagnostic 
assessments of students; interestingly, 10 percent of the teachers indicate that such assistance is 
not available in their school, which is not what one might hypothesize to be characteristic of RF 
schools. This may reflect the fact that in many schools, teachers do not administer diagnostic 
tests; therefore, in these schools, reading coaches do not provide teachers with this kind of 
support. 

To summarize the kinds of support teachers receive, we constructed a composite measure. 

Overall Support; that includes five supporting activities particularly salient to Reading First: 1) 
interpretation of assessment data, 2) classroom demonstrations, 3) grade-level meetings devoted 
to reading, 4) coaching or mentoring from a reading coach, and 5) use of assessment data to 
identify skills that need additional instruction or practice. Scores on Overall Support range from 
0-5, reflecting one point for each type of support teachers reported having received. On average, 
RF teachers reported receiving more types of direct support for their reading instruction than did 
teachers in Title I schools (4.1 vs. 3.5). 

Summary 

Professional development focused on scientifically based reading practices is central to the 
Reading First program. The objective of this professional development is to strengthen teachers’ 
knowledge about reading instruction and thereby improve their teaching quality and 
effectiveness. The findings provide consistent evidence that professional development for 
multiple aspects of reading instruction have been provided to RF principals, reading coaches, and 
teachers. This assistance includes both professional development related to instruction in the 
five dimensions of reading instruction as well as about overall teaching strategies. Further, the 
findings illuminate striking differences in professional development practice between RF and 
Title I schools. 

Significantly more Reading First teachers reported attending professional development 
workshops related to reading than did non-RF Title I teachers (94 percent vs. 81 percent). RF 
teachers, on average, reported having spent significantly more time attending professional 
development activities — conferences, workshops, college courses — in the past year than did 
teachers in non-RF Title I schools (40 hours vs. 24 hours). 

Professional development activities attended by RF teachers, as compared with non-RF Title I 
teachers, were significantly more likely to: 

• Offer incentives for participation, such as stipends (40 percent vs. 20 percent), release 
time (43 percent vs. 33 percent), or graduate credits (25 percent vs. 14 percent). 

• Be conducted by well-established and experienced trainers (75 percent vs. 65 
percent). 

• Require teachers to attend (74 percent vs. 57 percent). 

• Use a team-based approach (67 percent vs. 53 percent). 
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RF principals, reading coaches and teachers reported having participated in significantly more 
professional development activities to improve their knowledge of the five dimensions of reading 
instruction than did Title I teachers. For example, significantly more RF than Title I teachers 
received professional development in phonemic awareness (90 percent vs. 70 percent). RF 
teachers rated themselves (on a five-point scale) as significantly better prepared to teach the 
following skills than did Title I teachers: phonemic awareness, 4.13 vs. 3.66; decoding, 3.86 vs. 
3.35; vocabulary, 3.79 vs. 3.40; comprehension, 3.80 vs. 3.54; and fluency, 3.83 vs. 3.40. 

RF teachers were significantly more likely than teachers in Title I schools to report having 
received professional development assistance on administering and using assessments (85 
percent vs. 67 percent), interpreting assessment data (90 percent vs. 68 percent), and using 
diagnostic tests to guide instruction (66 percent vs. 49 percent). 

Reading coaches and principals also received professional development to support them in their 
roles. RF reading coaches were significantly more likely than their non-RF Title I counterparts 
to report having received professional development assistance on how to help teachers make 
reading instruction systematic and explicit (78 percent vs. 52 percent), and on the essential 
components of SBRR instruction (90 percent vs. 60 percent). Both of these topics are central to 
the Reading First program. 
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Chapter 8: Summary and Conclusions 



This interim report of the Reading First Implementation Evaluation addresses the following 
questions: 

1. How is the Reading First program implemented in districts and schools? 

2. How does reading instruction differ in Reading First schools and non-grantee Title I 
schools? 

The findings presented in this interim report provide some initial evidence that the Reading First 
Program is being implemented in districts and schools as intended by the legislation, specifically, 
in terms of the implementation of effective K-3 reading programs in schools. Below, we discuss 
each of these findings in more detail. 

Reading Instruction in Reading First Schools and Non-RF Title I 
Schools 

Reading First schools appear to be implementing the major features of the program as intended 
by the legislation. States and districts have been providing appropriate supports, particularly in 
terms of professional development related to reading, and the selection and use of assessments to 
inform instruction. Reading First schools have established instructional environments to support 
SBRR-based reading instruction, providing adequate time for reading instruction and using 
effective instructional materials and strategies. 

The reading programs being implemented in K-3 classrooms in Reading First and non-RF Title I 
schools appear to be different on a variety of components as follows: 



• Instructional Time. More time was spent teaching reading in RF schools than in 
Title I schools. Virtually all RF schools (98 percent) had a reading block (i.e., a time 
period that was formally scheduled for teaching reading) at each of grades K through 
3, significantly more than the 92 percent of classrooms in grades 1-3 and 88 percent 
of kindergartens that had reading blocks in Title I schools. Reading First schools 
reported, on average, a significantly longer reading block than did Title I schools (by 
about 15 minutes per day). Further, teachers in RF schools reported that, on average, 
they spent significantly more time on reading than did teachers in Title I schools — a 
difference of about 19 minutes per day, or almost 100 minutes per week. Teachers in 
RF schools were also significantly more likely than teachers in non-RF Title I schools 
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to report that they had increased the amount of time spent on reading from the 2003- 
04 to the 2004-05 school years (61 percent vs. 35 percent). 

• Instructional Materials. Reading First schools were significantly more likely than 
Title I schools to have changed the materials used for reading instruction by adopting 
a new core reading program (39 percent vs. 16 percent), a new program for struggling 
readers (74 percent vs. 43 percent), or a new reading assessment (71 percent vs. 36 
percent). Reading instructional activities and strategies in RF schools appear to be 
aligned with the tenets of the Reading First program. Further, RF teachers across all 
grades (K-3) rated materials as central to their instruction a greater proportion of 
SBRR- aligned practices than did teachers in Title I schools: These statistically 
significant differences were modest in size (e.g., second grade: 76 percent vs. 72 
percent), although the difference for kindergarten teachers was somewhat larger (77 
percent vs. 68 percent). 

• Instructional Strategies. RF teachers rated as central to their instruction a 
significantly higher proportion of scientifically based practices related to instruction 
the five dimensions of reading than did teachers in Title I schools: For example, 1) 

RF kindergarten and first-grade teachers rated as central to their instruction a 
significantly higher proportion of scientifically based practices for teaching phonemic 
awareness and decoding than did teachers in Title I schools; and 2) RF third- grade 
teachers rated as central to their instruction a significantly higher proportion of 
scientifically based practices in teaching vocabulary and fluency than did their Title I 
counterparts. However, there were no differences between RF and Title I teachers in 
their centrality ratings related to teaching comprehension skills. 

• Assessments. Reading First schools received significantly more external, 
nonfinancial assistance than did Title I schools in selecting assessment instruments 
(76 percent vs. 56 percent) and interpreting assessment results (82 percent vs. 70 
percent). RF teachers were significantly more likely to report having regularly 
scheduled, formal time set aside to use assessment data to plan instruction (84 percent 
vs. 74 percent). Across different types of direct support activities, a significantly 
larger proportion of RF teachers reported receiving assistance in administering 
assessments and using the information to guide instruction (85 percent vs. 67 
percent); interpreting assessment data (90 percent vs. 68 percent), and using 
diagnostic tests (66 percent vs. 49 percent), than did teachers in Title I schools. 

• Professional development. RF staff received significantly more professional 
development than did Title I staff. A significantly larger proportion of RF teachers 
reported having attended reading-related professional development workshops than 
did Title I teachers (94 percent vs. 81 percent). Further, RF teachers reported that, on 
average, during the last year, they spent significantly more hours attending these 



For this comparison, we used the newly funded RF schools since the mature RF schools may have already 
increased their instructional time in their first year of implementation, 2002-03. That said, about 45 percent of 
the mature RF schools did increase their instructional time in the 2004-05 school year. 
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professional development activities than did Title I teachers (40 hours vs. 24 hours). 
RF teachers were more likely to have received professional development in 
instruction in the five dimensions of reading as well as in overall teaching strategies. 
Indeed, RF teachers reported feeling better prepared to teach the five dimensions of 
reading than their Title I counterparts. 



These findings provide some initial evidence to suggest that the reading programs being 
implemented in grades to K-3 students in RF schools are meeting the objectives of the Reading 
First Program. 

Limitations 

As described in Chapter 1, there are several limitations to the findings presented in this report. 
First, the data sources (primarily surveys) represent respondents’ self-reported subjective 
perceptions and judgments about the implementation of their reading programs. Second, the 
Reading First legislation requires states to provide professional development in scientifically 
based reading instruction to all K-3 teachers, not only the teachers in schools that receive RF 
funding; this may reduce the potential to find large differences between RF and non-RF Title I 
schools. Third, many states require RF schools, not Title I schools, to have a reading coach; in 
fact, reading coaches in Title I schools are classroom teachers or reading specialists. We 
addressed this shortcoming by limiting the Title I group of reading coaches to those respondents 
who reported that they routinely engage in the activities used to define the responsibilities of a 
reading coach. Nevertheless, any observed differences between RF and reading coaches in 
Title I schools should be interpreted with somewhat more caution than observed differences 
between other respondent categories. 

The fourth limitation concerns the comparability of the Title I school sample. We constructed 
this sample purposefully to provide a context within which to understand the implementation of 
reading programs in Reading First schools. Comparisons between RF and Title I samples can be 
made, yet because the Title I school sample was not matched in any way to the RF school 
sample, it cannot be assumed to be equivalent to RF school sample. Although we have reported 
significant differences between the reading programs in RF and Title I schools, we must exercise 
caution when interpreting these observed differences. These differences cannot be attributed to 
the RF program. 

Future Activities 

This interim report presented findings primarily based on the spring 2005 survey data collection 
from principals, teachers, and reading coaches. The second wave of data collection will be 
conducted in 2007 and will provide information to address several questions not answered in this 
report: 
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• How does reading instruction differ in RF schools and non-grantee Title I schools as 
RF schools’ implementation efforts mature over time? 

• Does student achievement improve in schools with Reading First funds? 

• Is there any relationship between how schools implement Reading First and changes 
in reading achievement? 

The findings will be included in the final report to be published in the summer of 2008. 
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Appendix A: The Study Samples and the Sampling 
Weights 

The School Sample 



The study’s evaluation questions require sampling from three distinct groups of schools: (1) 
new RF schools (schools funded in January 2004 or later), (2) mature RF schools (schools 
funded on or before Dec. 31, 2003), and (3) non-Reading First Title I, schoolwide project 
(SWP) schools — schools in which at least 40 percent of the students are eligible for free or 
reduced-price lunches. We limited the Title I sample to SWP schools because that is the 
population from which Reading First schools are typically drawn. Each of these three groups 
was sampled to yield a nationally representative sample. To identify the populations of 
Reading First and Title I schools from which to select the study samples, we constructed 
sampling frames using data provided by the U.S. Department of Education.! Schools that did 
not have at least three of the target grades (K-3) were excluded. This section describes the 
selection of Reading Eirst and 550 Title I schools. 

Reading First Schoois 

We sampled and subsequently recruited equal numbers (550) of two types of Reading Eirst 
schools — newly funded, and mature Reading Eirst schools. These two samples are large 
enough to compute subgroup population level estimates. The two samples (new and mature) 
represent schools in different phases of implementation; this designation was based on 
Reading Eirst Program guidelines to states for their annual performance reports specifying 
that for “schools receiving grants between July 1 and December 31 of any reporting period, 
the current school year will be considered in the first year of implementation.” Therefore, 
schools awarded sub-grants before Dec. 31, 2003 were designated as mature schools for the 
spring 2005 data collection, when they were at least in their second year of implementation. 
All other RE schools were classified as new. 

In order to ensure that the new and mature samples of RE schools were representative of their 
respective populations, we stratified each group of RE schools into four census regions and 
four levels of school size, in which size represents the number of students in each school; this 
process created 16 strata. All Reading Eirst schools under the jurisdiction of the Bureau of 
Indian Affairs (BIA) were included in our sample in a 17th stratum. We sorted the schools 
within each stratum by state and urbanicity (four levels) to ensure that each sample provides 



* The Southwestern Educational Development Laboratory is maintaining a database for the U.S. Department 
of Education that tracks awards of Reading Eirst grants and sub-grants to states, districts, and schools. 

Also, the U.S. Department of Education is conducting a national evaluation of Title I schools and has 
provided us with the sampling frame used on that study. 
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a systematic representation on these demographic characteristics. Finally, we selected a 
systematic sample of schools from each stratum. 

Exhibit A. 1 summarizes the recruitment of the RE school sample. As of mid-October 2004, 
3,91 1 RE schools nationwide met the criteria for inclusion in the sample frame. Of these 
schools, we then selected 1,143 schools for recruitment into the study. A sample of 1,098 
schools agreed to participate in the study. Most schools (1,033, or 94 percent) have all four 
target grades. 



Exhibit A.1 



Recruitment of Reading First Schoois Based on October 2004 Data 




^ The population of RF schools is based on a database maintained by Southwestern Educational 

Development Laboratory. Abt obtained this database in October 2004, in time to construct the sampling 
frames, then select and recruit the samples of RF schools. The complete population of RF schools at that 
time was 4,352; however, schools were removed from the frame for one of five reasons; 1) if they did not 
have three of the four target grades; 2) if they are in the RF Impact Study; 3); if they are private schools, 
both because the composition of the student body is qualitatively different (i.e., primarily self selected) than 
the composition of students in public schools, and because program operations in private schools are not 
held to the same policies as program operations in public schools; 4) if they are located in American 
Samoa; or 5) they were no longer RF schools. 
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After conducting the data collection, we used principals’ survey responses to verify and 
correct, if necessary, the classification of RF schools as new or mature. Approximately 88 
percent of these schools were correctly classified as new or mature. Most classification errors 
reflected inaccurate designations as mature; principals’ survey responses showed that 
because their schools were in their first year of implementation, they should be classified as 
“new.” 

Title I Schools 

We recruited a nationally representative sample of 551 non-Reading First Title I, SWP 
schools. To identify the appropriate population of Title I schools, we relied on the Common 
Core of Data (CCD) — a database that contains relevant demographic information on all 
schools nationwide. At the time we drew the sample, the CCD included data from the 2002- 
03 school year. 

Exhibit A. 2 summarizes the recruitment of the Title I school sample. A total of 14,684 
Title I, SWP schools had at least three of the four target grades. We constructed the same 16 
strata (geographic region by school size) used for the RF school sample, then sorted by state 
and urbanicity. For the purpose of selecting the Title I sample, we did not create a separate 
stratum for the BIA schools; rather, BIA schools were distributed across the 16 strata. We 
selected 718 Title I schools for potential recruitment into the study. A small percentage of 
schools were ineligible because they a) did not have at least three target grades, b) were 
closed, or c) were no longer Title I schools. About 9 percent, or 67 schools, refused to 
participate in the study. Additionally, 53 schools, despite repeated communication, never 
explicitly either refused or agreed to participate in the study. We successfully recruited 551 
schools into the study; most (91 percent) have the four target grades. 
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Exhibit A.2 

Recruitment of Titie i Schoois 



Population of 
Title I Schools 
14,684 




Source: Data Collection Receipt Tracking File. 



Data from new and mature RF schools allows us to answer the descriptive question about 
Reading First implementation in districts and schools. Data from mature RF schools and 
Title I schools allows us to compare the reading programs implemented in these groups of 
schools. 

The Principal, Reading Coach, and Teacher Samples 

We obtained complete teacher rosters (grades K-3) from each school successfully recruited 
into the study. Each school’s principal was included in the evaluation. Similarly, all reading 
coaches, typically one per school, were included in the evaluation sample. 

To construct the teacher sample, we randomly selected one teacher in each of grades K-3. 
Some study schools have fewer than four teachers in the sample because the school only has 
three of the four target grades (i.e., a K-2 school), or because the school has only combined 
classrooms (i.e., K-1, 1-2, 2-3). Exhibit A. 3 displays the teacher and reading coach samples 
for the 1,649 schools recruited into the study. In total, there are 6,466 teachers and 1,346 
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reading coaches in the sample of recruited schools. Almost all (99 percent) of the RF schools 
have a reading coach. Only half (52 percent) in the Title I schools designated someone as a 
reading coach. 



Exhibit A.3 

Principai, Reading Coach, and Teacher Sampies 



Potential Respondent Samples 



Type of 
Respondent 


Reading First 
Schools 


Title 1 Schools 


Total 


Teachers 








Kindergarten 


1,081 


542 


1,623 


1st grade 


1,088 


545 


1,633 


2nd grade 


1,087 


541 


1,628 


3rd grade 


1,058 


524 


1,582 


Total teachers 


4,314 


2,152 


6,466 


Reading coaches 


1,060 


286 


1,346 


Principais 


1,098 


551 


1,649 


Source: Data Collection Receipt Tracking File. 



Response Rates. All but six RF schools and 10 Title I schools (1,092, and 541, respectively) 
returned at least one survey. Exhibit A.4 presents response rates at the school and individual 
respondent levels. About 88 percent of the schools returned completed surveys for all the 
selected respondents within their schools; another 8 percent of schools completed all but one 
of the surveys and only 17 schools, or 1 percent, were nonrespondents, returning no surveys. 
Response rates were slightly higher for Reading First schools compared to Title I schools; 97 
percent of Reading First schools were either complete or missing only one survey, compared 
to 94 percent for Title I schools. It is important to note here, that Reading First schools are 
required, as part of their acceptance of a sub-grant to participate in a national evaluation. No 
corresponding requirement existed for the Title I schools. 



The response rate across all types of respondents and all schools is 96 percent. Of 9,460 
potential respondents, 9,076 individuals returned completed surveys. For teachers in 
Reading First schools response rates are approximately 96 percent across the four grade 
levels, compared to 94 percent in Title I schools. 
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Survey Data Coiiection Response Rates for Reading First and Titie i Schoois, 



2004-05 School Year 






Total 


Reading First 




Title 1 




Response Rate 


Response Rate 


Response Rate 




Number 


Percent 


Number 


Percent 


Number 


Percent 


Response Status 














Complete 


1,441 


88% 


978 


89% 


463 


84% 


Almest cemplete 


139 


8 


85 


8 


54 


10 


Partial 


53 


3 


29 


3 


24 


4 


Nonrespendents 


16 


1 


6 


1 


10 


2 


Total 


1,649 


100 


1,098 


100 


551 


100 


Type of Respondent 














Principal 


1,574 


95% 


1,057 


96% 


517 


94% 


Teacher — K 


1,563 


96 


1,054 


98 


509 


94 


Teacher — 1st 


1,557 


95 


1,043 


96 


514 


94 


Teacher — 2nd 


1,559 


96 


1,048 


96 


511 


94 


Teacher — 3rd 


1,506 


95 


1,013 


96 


493 


94 


Reading coach 


1,318 


98 


1,045 


97 


272 


95 


Total 


9,076 


96 


6,260 


97 


2,816 


94 



Source: Data Collection Receipt Tracking File. 



Sampling Weights 

School-level Weights 

The construction of the school weights is based on the complete recruitment sample of 1,861 schools 
(1,143 Reading First and 718 Title I schools). Based on the stratification descrihed above, schools 
from each stratum were weighted to represent that stratum’ s population of schools and then adjusted 
for nonresponse and ineligibility. This school-level weight was constructed for the 1,633 schools 
(1,092 RF and 541 Title I) that returned at least one survey. The same method was used to construct 
weights for principals and for reading coaches. Because we have principal surveys from 1,574 rather 
than 1,633 schools, and reading coach surveys from 1,318 schools rather than 1,633 schools, these 
weights were adjusted for non-response at the principal and reading coach levels. 

Teacher-level Weights 

We constructed teacher weights for these analyses for two reasons. First, some questions on the 
teacher surveys ask about classroom rather than school activities, and because we have a national 
sample of classrooms in RF and Title I schools, we want to be able to generalize to this classroom 
population. Second, we have only 1 teacher respondent per grade per school; this teacher represents 
the population of teachers in that school at that grade. That population can vary substantially — from 
one teacher to 10 or 12 teachers. If we applied the school weights to teacher responses, all teachers’ 
responses would have equal weight, regardless of the actual numbers of teachers at that grade level. 
For these reasons, we constructed a set of teacher weights that allow us to generate estimates for the 
population of RF and Title I teachers at each of the four target grades (K-3). 
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The teacher weights were constructed to represent the number of teachers in their school at their 
grade level. These weights were also adjusted to include teachers who teach multiple grades in their 
school. The “within-school” teacher weights are multiplied hy the school-level weight in order to 
represent the population of teachers across all RF and Title I schools. For example, if a first grade 
teacher in our sample were in a school with three other first grade teachers, her ‘within-school’ 
weight would he four. If the school-level weight for this school were five, then teacher-level weight 
would he 20 (five * four). As shown in Exhibit A.5, the study sample of RF teachers represents over 
65,000 teachers in the population of RF schools, and the Title I teacher sample represents 203,659 
teachers. 



Exhibit A.5 

Weighted Estimates of the Popuiation of Teachers in Reading First and 



Titie i Schoois, by Grade 



Weighted Number of Teachers 


Grade Levei 


Reading First 


Title 1 


Kindergarten 


15,780 


49,453 


1st grade 


17,385 


54,327 


2nd grade 


16,396 


51,540 


3rd grade 


15,685 


48,339 


Total 


65,246 


203,659 



Source: Abt Associates SAS Analytic datasets. 
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Aht ID# // Barcode here. . .. 






0MB Number: 1875-0232 

Expiration Date: 10/31/07 





Reading First impiementation Study 
Grade 1 Teacher Survey 



The U.S. Department of Education’ s Policy and Program Studies Service has contracted with Aht 
Associates Inc. to conduct a national evaluation of K-3 reading instruction in Reading First and Title I 
schools. A sample of 1 100 Reading First schools and a comparison group of 550 Title I schools, have 
been selected to participate in this study. The principal, reading coach, and a selected sample of K-3 
teachers from each participating school are being asked to complete a survey. Participants will help 
inform the U.S. Department of Education, Congress, policymakers, practitioners, and researchers about 
how K-3 reading instruction is implemented in schools and what strategies teachers use to provide high- 
quality, evidence-based reading instruction in grades K-3. 
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Additional Information 



The survey will take you approximately 30 minutes to complete. Although your participation in the 
survey is entirely voluntary, we strongly encourage you to participate. There are no joh-related or other 
consequences for not participating. You may also choose to answer some questions on the survey and not 
others — although we urge you to complete as many questions as possible. 

All responses to the survey will he kept confidential. All individual identifying information will he used 
only hy persons on the research team. Information such as school location (state), participants’ general 
joh titles, grades they teach, and gender will he included in the study data files to he submitted to the 
Department of Education. However, participants’ names will be stripped from all analysis data files and 
data files to be submitted to the Department of Education. We will not report any data about individual 
classrooms — all information will be reported at the grade and school levels. Neither your school nor your 
district will have access to any of the completed surveys at any time. 

Please return your completed survey to Abt Associates in the enclosed stamped, pre-addressed mailing 
envelope. If you decline to participate, simply do not return the survey in the mail. 

Thank you for your cooperation with this survey! 



According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of 
information unless such a collection displays a valid OMB control number. The valid OMB control 
number for this information collection is 1875-0232. The time required to complete this information 
collection is estimated to average 30 minutes per response, including the time to review instruction, 
search existing data resources, gather the data needed, and complete and review the information 
collection. If you have any comments concerning the accuracy of the time estimate or suggestions for 
improving this form, please write to; Policy and Program Studies Service, U.S. Department of 
Education, 400 Maryland Avenue, SW, Washington, DC, 20202. 



Instructions 

Unless otherwise noted, your responses should reflect your experiences during the 2004-2005 school 
year in the school to which this survey was sent. 



■ Please complete all questions; each question includes directions for recording your answer. 

■ You are sometimes told to skip over some questions in the survey. When this happens, you will 
see an arrow with a note that tells you what question to answer like this: 

□ i Yes 

□2 No — > Skip to E4 



■ If you have any questions about how to complete the survey, please call: 

. This is a free call and will connect you with our expert interviewers 



who can assist you. 
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A. Your Background and Experience 



Al. Including this year, how long have you been a teacher? If less than one year, please enter ‘ 1 ’ 





Enter # of years below. . . 


a. Total number of years as a teacher 


vears 


b. Number of years teaching in grades K-3 


vears 


c. Number of years teaching at this school 


vears 


d. Number of years teaching reading 


vears 


(either separately or as part of regular classroom instruction) 



A 2 . What grade(s) are you currently teaching? (Check all that apply) 

Q 1 Kindergarten 

□ 2 First grade 

□ 3 Second grade 

□ 4 Third grade 

Q95 Other (Please specify): 

A 3 . Descrihe your certification status. Which of the following descrihes the teaching certificate you 
currently hold in this state? (Check one) 

Q 1 Regular or standard state certificate or advanced professional certificate 
Q2 Prohationary certificate (issued after satisfying all requirements except the completion of a 
probationary period). 

□ 3 Provisional or other type of certificate given to persons who are still participating in what the 

state calls an “alternative certification program.” 

Q4 Temporary certificate (requires some additional college coursework, student teaching, and/or 
passage of a test before regular certification can be obtained). 

□ 5 Waiver or emergency certificate (issued to persons with insufficient teacher preparation who 

must complete a regular certification program in order to continue teaching). 

□5 I do not have any of the above certifications in this state. ^ Skip to A 4 

A 3 a. If certified, identify the area(s): (Check all that apply) 

Q 1 Elementary education 
Q2 Early childhood education 
Q3 Reading 
Q4 Special education 
Q5 Bilingual / ESOE/ESE education 

Q95 Other (Please specify): 
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A4. What is the highest degree you have obtained as of December 2004? (Please check one) 

□ 1 Bachelors 

□ 2 Bachelors + additional courses 

□ 3 Masters 

□ 4 Masters + additional courses 
Q5 Doctorate 

Q95 Other (Please specify): 

A5. How well do you feel your pre-service teacher training prepared you to teach each of the 
following dimensions of reading? 



Pre-service teacher training refers to training you received before you became certified and began 
teaching. For those who began their teacher career through an alternative certification or emergency 
certification program, and began teaching before they were certified, pre-service teacher training refers 
to the training you received to become fully certified. 



Please choose a ‘ 1 ’ if you were ‘not at all prepared’ to teach the dimension and a ‘ 5 ’ if you were 
‘extremely well prepared.’ 
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B. 



Characteristics of Your Students 



In answering the remainder of the survey, include ALL of the students to whom you teach reading, 
whether you teach reading on your own in a self-contained classroom, to a group that includes students 
from other classes, or to more than one group of students. 





Enter # below ... 


Bl. What is the total number of students to whom you currently teach reading? 


# 


B2. How many groups do these students represent? For example, if you teach all of these students 
during a single reading block you should consider them one group. If you teach some students in 
one reading block, and the rest of the students in another reading block, you should consider 
them two groups. 


# 


B3. How many of these students are English Language Learners (ELLs), also referred to as Limited 
English Proficient (LEP) or English for Speakers of Other Languages (ESOL) students? 

(e.g., Spanish, Portuguese) 


# 




Enter # below ... 


B4. Number of ELL students whose home language is Spanish. 


# 


B5. How many of your reading students are also special education students with lEP’s who receive 
special education services in reading? 


# 


B6. How many of your reading students receive intervention services in reading from you or another 
teacher or tutor? Reading Intervention is a program designed for struggling readers to be 
used only with straggling readers in addition to the core-reading program. 


# 


B7. How many of your students are reading at or above the approximate level expected for their 
grade? 


# 
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c. 



Instruction and Assessment in Reading 



Cl. Last week, approximately how many minutes per day did you devote to reading instruetion? 

Include only reading instruction and not other language arts such as writing, spelling. Fill in the 
chart for each day last week with your best estimate of the number of minutes. . . 



Monday 


Tuesday 


Wednesday 


Thursday 


Friday 


# min. / dav 


# min. / dav 


# min. / dav 


# min. / dav 


# min. / dav 













C2. Has the average number of minutes you spend each day this year teaching reading increased, 
remained the same, or decreased from last year (2003-2004)? (Please check one) 

I did not teach reading last year 

□ 2 Decreased 

□ 3 Remained the same 

□ 4 Increased 



C3. How often during this school year is time regularly scheduled and formally set aside during the 
school day for Grade 1 teachers to: 





Check only one box for each item ... 




Not at all 


1-4 times 


5-8 times 


Once a 
month 


Once a 
week or 
more 


Occurs 

only 

informally, 
as needed 


a. Collaborate on reading lesson planning and 
instraction. 






□ 3 


□4 


□5 




b. Observe reading instraction in other 
classrooms. 










□5 




c. Use assessment data to plan instruction. 






□ 3 


□4 


□5 


□6 


d. Participate in coaching with or be coached 
about reading by other teachers. 






□ 3 


□4 


□5 


□6 


e. Be coached about my reading instraction by 
a reading coach (see below). 






□ 3 


□4 


□5 


□6 


f. Coordinate reading interventions for 
straggling readers with special education 
staff. 






□ 3 


□4 


□5 


□6 


g. Coordinate reading interventions for 
struggling readers with ELL staff. 






□ 3 


□4 


□5 


□6 







NOTE: A reading coach is a staff member whose primary role is to provide ongoing training and support to 
classroom teachers in the delivery of effective reading instraction. This assistance may include planning instruction, 
providing demonstration lessons, observing and providing feedback, using assessment results to guide instraction, etc. 



B-8 



Reading First Implementation Evaluation: Survey of Grade 1 Teachers 



Appendix B 

















C4. Please describe your use of the following reading instructional activities this year. 

• Check column A ONLY if the instructional activity is one that you use frequently when you teach 
reading or one on which you rely heavily in your reading instruction. 

• Check column B if you use the instructional activity, but it is a small part of your teaching, and not 
one you use frequently. It might be an activity that you use if there is time, but it is not one on which 
you rely heavily for your reading instruction. 

• Check column C if the activity is not one you use in your reading instruction. 







Check only one box for each item... 






A 

Central to mv 
reading 
instruction 


B 

Small part of 
my reading 
instruction 


c 

Not Part of my 
reading 
instruction 




a. I provide feedback on errors as students read orally. 


□i 


□2 


□ 3 


Reading 

text 


b. Students read texts that are easy to decode. 




□2 


□ 3 


c. Students read silently. 


□i 


□ 2 


□ 3 




d. Students reread familiar stories. 




□ 2 


□ 3 




e. Students select books from the library for 
independent reading. 


□i 


□ 2 


□ 3 




f. I develop language experience stories with my class. 




□ 2 


□ 3 




g. Pairs of students read aloud together. 


□i 


□ 2 


□ 3 


Reading 

text 


h. Students read aloud with expression and proper 
phrasing. 




□2 




i. Students reread to find facts to answer questions. 




°2 






j. Class creates story maps. 




□ 2 


□ 3 




k. I listen to students read aloud without correcting 
errors. 


□i 


□ 2 


□ 3 




1. Students isolate sounds in words that I say. 




□ 2 


□ 3 




m. Students practice naming letters. 


□i 


□ 2 


□ 3 




n. Students blend phonemes to form words. 




□ 2 


□ 3 


Work 


o. Students practice reading high frequency words for 
automaticity. 


□i 


□ 2 


□ 3 


with 

sounds 


p. Students use knowledge of root words, prefixes, and 
suffixes to decode new words. 




□ 2 


□ 3 


and 

words 


q. I stop students while reading and have them self- 
correct misidentified words. 


□i 


□ 2 


□ 3 




r. Students use pictures to identify unknown words. 




□ 2 


□ 3 




s. I teach decoding skills while reading stories. 




°2 


°3 




t. Students practice writing words as separate syllables. 




□ 2 


□ 3 




u. I teach decoding skills with word families. 


□i 


□ 2 


□ 3 
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C4. CONTINUED. Please describe your use of the following reading instructional activities this 

year. 

• Check column A ONLY if the instructional activity is one that you use frequently when you teach 
reading or one on which you rely heavily in your reading instruction. 

• Check column B if you use the instructional activity, but it is a small part of your teaching, and not 
one you use frequently. It might be an activity that you use if there is time, but it is not one on which 
you rely heavily for your reading instruction. 

• Check column C if the activity is not one you use in your reading instruction. 







Check only one box for each item... 






A 

Central to 
my reading 
instruction 


B 

Small part of 
my reading 
instruction 


c 

Not Part of my 
reading 
instruction 




V. I engage students in rhyming games and songs. 






□ 3 




w. Students retell stories in sequence and identify 
characters and main events. 




□2 


□ 3 


Other 

Techniques 


X. I read stories aloud to students. 






□ 3 


y. Students write stories using invented spelling. 


□i 


□2 


□ 3 




z. I discuss new and unusual words before reading. 




□ 2 


□ 3 




aa. Students write vocabulary words in sentences. 




°2 






bb. Students read stories they have written to others. 


□i 


□ 2 


□ 3 




cc. Students make predictions while reading stories. 




°2 






Id. Students use dictionaries to find word meanings. 




□ 2 


□ 3 


Other Skills 


ee. Students are given time to read on their own for 
enjoyment. 




□ 2 


□ 3 




Ff. Students develop questions about text material. 




□ 2 


□ 3 




gg. Students act out story as a play. 




□ 2 


□ 3 



C5. Have you eliminated any activities in the above lists that you did last year? If so, which ones 
have you eliminated? 



(Please identify by letter) 
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C6. Please describe your use of the following teaching strategies and materials this year. 

• Check column A ONLY if the item is one that you use frequently or one on which you rely heavily in your reading 

instruction. 

• Check column B if you use the item, but it is a small part of your teaching, and not one you use frequently. It may be 



an approach you use if there is time, but it is not one on which you rely heavily. 
• Check column C if the item is not one you use in your reading instruction. 





Check only one box for each item... 




A 


B 


c 




Central to 


Small part of 


Not Part of 




my reading 


my reading 


my reading 




instruction 


instruction 


instruction 




a. Provide time in reading block for skill practice on own. 




□2 


□ 3 




b. Provide materials for at-home practice of skills introduced in 


n 


n 


n 




class. 


' 1 


'—‘2 


'—‘3 


Instruction 


c. Provide extra reading instructional time for straggling 
students. 




□2 


□3 




d. Include writing opportunities in reading instruction. 




□2 


□3 




e. Build spelling practice into reading instruction. 




□2 


□3 




f. Develop reading skills through science and social studies. 




°2 






g. Teach whole class reading lessons. 




°2 






h. Work one-to-one with students on reading. 




□2 


□ 3 




i. Work with small groups of students. 




□2 


□ 3 


Grouping 


j. Group students based on skill levels. 




□2 


□ 3 




k. Group students based on mixed abilities (cooperative 










groups). 


' 1 


'—‘2 


' 3 




1. Pair strong readers with those with weaker skills. 




°2 


^3 




m. Use core reading series. 




□2 


□ 3 




n. Use supplementary reading materials. 




□2 


□ 3 




o. Use children’s trade books. 




□2 


□ 3 


Reading 


p. Use books that are easy to decode. 




□2 


□ 3 


materials 


q. Use books with patterned predictable language. 




°2 


^3 




r. Use separate intervention materials for some students. 




□2 


□ 3 




s. Use reading software/technology. 




□2 


□ 3 




t. Use teacher-made materials. 




□2 


□ 3 




u. Use test results to organize instructional groups. 




□2 


□ 3 




V. Use informal reading inventories. 




□2 


□ 3 




w. Use tests to determine progress on skills. 




°2 


^3 


Assessments 


X. Use tests to determine who can benefit from the core 
reading series. 




□2 


□ 3 


y. Use diagnostic tests to identify students who need reading 










intervention services. 


— ^ 1 


— 2 


' — ' 3 




z. Use screening tests to identify students who need a 
supplementary reading program. 




°2 


^3 




aa. Conduct miscue analysis, analyzing errors students make 
while reading aloud. 




°2 


^3 
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C 7 . Which specific formal or informal assessment(s) do you find most useful for the following 
purposes? 





List below up to three names of assessments for each purpose... 




Assessment 1 


Assessment 2 


Assessment 3 


a. Placement and/or grouping students 


1 


2 


3 


b. Determining student mastery of skills 


1 


2 


3 


c. Identifying the core deficits of 
struggling students 


1 


2 


3 



C8. What materials are used with English Language Learners (ELLs) to whom you teach reading? 

(Check all that apply) 

□ 1 Do not teach ELLs — >Skip to C 9 

□ 2 Core reading program materials in the native language of the ELL 

□ 3 ELL students use the same materials as other students 

□ 4 Core reading program materials, plus supplementary /intervention resources written in the 

ELL’s native language 

□ 5 Core reading program materials, plus supplementary /intervention resources written in English 

especially for ELLs 

□ 6 Alternative core reading program materials in English geared toward the instructional level of 

the ELL 

Q95 Other (Please specify): 

C 9 . What additional supports have students who are struggling readers received in the last month? 
Check whether or not your students who are struggling readers received each of the supports 
during the past month. 



Supports for Struggling Readers 


Check only one box for each 
item... 




Received 


Did not receive 


a. Diagnostic assessment to determine core deficits. 


□1 


□2 


b. Extra practice in the classroom with phonemic awareness. 




°2 


c. Extra practice in the classroom with decoding. 


□1 


□2 


d. Extra practice in the classroom with fluency. 




°2 


e. Extra practice in the classroom with comprehension. 




°2 


f. Extra instructional time. 




□ 2 


g. Placement in materials that supplement the core reading 
program. 


□1 


□ 2 


h. Placement in different level of core reading program. 




□ 2 


i. Placement in separate core reading program. 


□1 


□ 2 


j. Placement in special intervention program. 


□1 


□ 2 
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C9. 



CONTINUED. Additional supports have students who are struggling readers received in the last 
month? Check whether or not your students who are struggling readers received each of the 
supports during the past month. 



Check only one box for each item 



Supports for Struggling Readers 





Received 


Did not receive 


k. 


Work with tutor on one-to-one basis 






1. 


Work with reading specialist on one-to-one basis 






m. 


Work with reading specialist in small group. 






n. 


Work with more advanced peer. 






a. 


Special materials for parents to provide practice. 







CIO. What additional supports have students who are struggling readers received in the last month? 
Check whether or not your students who are struggling readers received each of the supports 
during the past month. 



Supports for Struggling Readers 


Check only one box for each item ... 


„ . . Did not 

Received 

receive 


N/A, no 
ELL’S 


a. If English language learner(s), English as a Second Language 
instmction. 






b. If English language leamer(s), provide reading instruction in home 
language. 


□ 1 Dj 


□s 


c. If English language learner(s), in classroom help in reading from 
ELL teacher. 


□ 1 Dj 


□s 



D. Professional Development in Reading for K-3 Teachers 



Dl. During the current school year, including summer 2004, in how many of each of the following 

types of professional development activities in reading have you participated? Please count each 
activity only once. What is the total number of hours you spent in these activities? 



First, write in the number of activities of each type in which you have been engaged. Then, write the 
total number of hours y ou spent in these activities. Mark 0 if you participated in none. 





Enter # below... 


#of 

Different 

workshops 


Total hours 


a. Attended short, stand-alone training or workshop in reading (half-day or less) 


# 


# 


b. Attended longer institute or workshop in reading (more than half-day) 


# 


# 


c. Attended a college course in reading (include any courses you are currently attending) 


# 


# 


d. Attended a college course in reading (include any courses you are currently attending) 


# 


# 


e. Attended a conference about reading (might include multiple short offerings) 


# 


# 
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D 2 . Below is a list of professional development activities that are often used to provide ongoing, 
direct support to teachers for teaching reading. 

• In the first column, please indicate whether you have received any of the following types of 
assistance/support for teaching during the current school year, including summer 2004 ? 

• If you did not receive that type of support, please indicate whether the support was available, 
hut you did not receive it (column 2 ), or if it was not available at your school (column 3 ). 





Check only one box for each item ... 




Types of assistance 
I received this year 


Available, but I 
did not receive 


Not available at 
my school 


a. Coaching or mentoring by reading coach in programs, 
materials, or strategies. 








□2 




^3 


b. Coaching or mentoring from fellow teacher. 








□ 2 




□ 3 


c. Peer study group or collegial circle for group study. 








□2 




^3 


d. Demonstrations in my classroom. 








□ 2 




□ 3 


e. Observations of other teachers. 








□ 2 




□ 3 


f. Diagnostic testing help from a reading coach or specialist 
for individual students. 








□ 2 




□ 3 


g. Intervention service help from a reading coach or specialist 
for individual students. 








□ 2 




□ 3 


h. Interpretation of assessment data. 








□ 2 




□ 3 


i. Grade level meetings devoted to reading. 








□2 




^3 


j. Using assessment data to determine topics that require 
additional instruction or practice. 








°2 




^3 


D 3 . During the current school year, including summer 2004 , 

reading professional development activities for K -3 t< 

most closely describes your professional development. 


approximately how many of the 
iachers? Please choose the category that 

Check only one box for each item... 






None 


One- One- Three- 

Quarter Half Quarters 


All 


a. require teachers to attend? 


□i 


□2 


□ 3 


□4 




b. were also attended by the principal? 




□2 


□ 3 


□4 




c. provide teachers options among which to choose? 


□i 


□2 


□ 3 


□4 




d. provide a stipend? 




□2 


□ 3 


□4 




e. provide follow-up activities? 


□i 


□2 


□ 3 


□4 




f. include release time for participating teachers? 




□2 


□ 3 


□4 




g. offer graduate college credits? 


□i 


□2 


□ 3 


□4 




h. are held in a convenient location (e.g., activities held at school)? 




□2 




°4 




i. use a team-based approach (joint training of people who work 
together)? 


□i 


□2 


□ 3 


□4 




j. are given by trainers or facilitators who have a well-established 
reputation? 




□2 




°4 


^5 
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D4. Below is a list of topics that are often covered in professional development activities designed to 
provide teachers with new information about the content of reading instruction. 

• In column A, identify the topics that were addressed in professional development activities in which 
you participated during the current school year, including summer 2004. 

• In column B, please identify 5 topics in which you would like more professional development, whether 
or not this school’s professional development activities have covered these topics. 

• Please check all that apply in column A, and 5 choices in column B. 



Professional development is defined as any activity in which a teacher has iearned about reading or 
reading instruction. This inciudes schooi-based workshops, meetings with reading coaches, and 
meetings with a study group of other teachers. 







CHECK ALL THAT 
APPLY 


INDICATE NO 
MORE THAN 5 
TOPICS 






A. Topics 


B. Topics in which 






addressed in 


I’d like more 






professional 


professional 






development 


development 




a. Building phonological awareness, e.g. rhymes, dividing 


n, 




Phonemic 

Awareness 


spoken language into sentences, words, syllables 




'—‘2 


b. Identifying, adding, deleting sounds in spoken words 








c. Blending phonemes to form words 




□2 




d. Teaching letter-sound correspondence 


□ 3 


□4 


Decoding 


e. Teaching letter patterns (blends, digraphs, diphthongs) 




□2 


f. Using syllable patterns to read words 


□ 3 


□4 




g. Teaching component parts: roots, prefixes, suffixes 




□2 




h. Teaching use of dictionary, thesaurus 


□ 3 


□4 


Vocabulary 


i. Direct teaching of vocabulary words and their meaning 




°2 




j . Antonyms and synonyms 


□ 3 


□4 




k. Teaching sight words 




□2 


Fluency 


1 . Guided oral reading 


□ 3 


□4 




m. Encouraging expression while reading 




°2 




n. Setting motivation/asking prediction/preview questions 


□ 3 


□4 




o. Constructing information about character, setting, and main 








events 


' ' 1 


' — ' 2 


Comprehension 


p. Summarizing main ideas in narrative and informational text 


□ 3 


□4 


q. Self-monitoring strategies 




°2 




r. Asking questions at different levels (literal, inferential) 


□ 3 


□4 




s. Strategies for organizing text structure, e.g. story maps 




□2 


Other 


t. Other topic in the dimensions of reading. (Please specify: ) 


°3 


°4 
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D5. Below is a list of topics that are often covered in professional development activities that are 
designed to provide teachers with new information about teaching strategies used during reading 
instruction. 

• In column A, identify the topics that were addressed in professional development activities in 
which you participated during the current school year, including summer 2004. 

• Then, in column B, please identify 5 topics in which you would like more professional 
development, whether or not this school’s professional development activities have covered these 
topics. 



• Please check all that apply in column A, and 5 choices in column B. 







CHECK ALL 
THAT APPLY 


INDICATE NO 
MORE THAN 5 
TOPICS 


Topic 




A. Topics 
addressed in 
professional 
development. 


B. Topics in which 
I’d like more 
professional 
development 




a. How to use the core reading program 




□2 




b. How to use children’s literature to teach reading 








c. How to use reading research to guide content of instruction 




□2 


Teaching 

Strategies 


d. How the core reading program incorporates research principles 




□4 


e. How to use the supplemental reading program(s) 




□2 




f. How to integrate reading and writing instraction 


°3 


□4 




g. Strategies for teaching reading to ELLs 




□2 


Grouping 


h. Learning styles 


□ 3 


□4 


i. How to organize small group instraction 




°2 




j. How to diagnose reading problems 


□ 3 


□4 


Assessment 


k. How to administer assessments 




°2 




1 . How to interpret and use assessment data to guide instraction 


□ 3 


□4 




m. How to help struggling readers with decoding 




□2 




n. How to help straggling readers with vocabulary 


□ 3 


□4 


Struggling 

Readers 


o. How to help straggling readers with comprehension 




□2 


p. How to motivate readers 


□ 3 


□4 




q. Strategies for teaching reading to students with diagnosed 
learning disabilities 


□1 


□2 




r. How to use state/district content standards for curriculum 
planning and teaching 


°3 


°4 


Organization/ 

planning 


s. How to align reading curriculum and instruction with 
state/district assessments 


□1 


□2 


t. How to work with parents 


□ 3 


□4 




u. Classroom management 




°2 




v. Other (Please specifvl: 


□ 3 


□4 
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D6. How well do you feel the professional development activities in which you participated during 
the current school year (including summer, 2004 ) prepared you to teach each of the following 
dimensions of reading? Please choose a ‘1’ if you were ‘not at all prepared’ to teach the 
dimension and a ‘5’ if you were ‘extremely well prepared.’ 




E1 . Support for Teaching Reading 



El. Who provides you with feedhack about your teaching of reading? (Check all that apply.) 

Q 1 Principal 

□ 2 Assistant Principal 

□ 3 School reading coach (staff member whose primary role is to provide ongoing training and 

support to teachers in the delivery of effective reading instruction) 

□ 4 Peer coach 

□ 5 Mentor teacher 

□5 District reading coach 

□ 7 External reading coach (e.g., university, regional, or state professional development or 

technical assistance provider) 

□ 8 Students in my class 

□ 9 No one provides feedback 

Q95 Other (Please specify): 

E 2 . Is there a school-based reading coach at your school? 

Qi Yes 
□2 No 
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(Check all that apply.) 



Check all that apply for each item 





Principal’s 

responsibility 


Reading Coach 
responsibility 


Other person’s 
responsibility 


Don’t 

Know 


a. Selects core reading materials 










b. Selects supplemental reading materials (for use with the 
whole class) 




□2 




□b 


c. Selects intervention materials (for use with struggling 
readers) 


□i 


□2 


□ 3 


□b 


d. Selects reading assessment instruments 




□2 


□ 3 


□b 


e. Monitors implementation of the reading program 


□i 


□2 


□ 3 


□b 


f. Models effective reading instructional strategies in 
classroom (demonstration lessons) 




□2 


□ 3 


□b 


g. Provides feedback to teachers about reading instruction 


□i 


□2 


□ 3 


□b 


h. Reviews individual students’ progress in reading 




°2 


°3 


□b 


i. Assists teachers in using reading assessment data to make 
instractional decisions 




°2 


°3 


°B 


j. Leads grade level team meetings for reading 




□2 


°3 


□b 


k. Reviews teachers’ reading lesson plans 


□i 


□2 


□ 3 


□b 



E4. The next set of statements is about your reading program. Please indicate the extent to which you 
agree or disagree with each statement. 



Check only one box for each item 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. I feel I need to make changes in the methods I use to teach children 
to read. 




□ 2 


□ 3 


□4 


b. Other faculty/staff members have helped me to understand the 
difficulties that some children have in learning to read. 


□1 


□ 2 


□ 3 


□4 


c. I have benefited from opportunities to learn more about methods 
for teaching reading. 




□ 2 


□ 3 


□4 


d. The children in my class are making satisfactory progress in 
learning to read. 


□1 


□ 2 


□ 3 


□4 


e. I do not have sufficient materials to teach reading effectively. 




□ 2 


□ 3 


□4 


f. I do not understand why some children learn to read easily while 
other children struggle to learn basic reading skills. 


□1 


□ 2 


□ 3 


□4 


g. The reading coach supports my efforts to teach reading effectively. 




□ 2 


□ 3 


□4 


h. I have a good understanding of how children acquire language and 
literacy skills. 


□1 


□ 2 


□ 3 


□4 


i. I wish I had more opportunities to discuss how to teach reading 
with other teachers. 




□ 2 


□ 3 


□4 
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E4. CONTINUED. The next set of statements is about your reading program. Please indicate the 
extent to which you agree or disagree with each statement. 

I Check only one box for each item ... 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


j. I know the current reading skill levels of all my students. 






□ 3 


□4 


k. I know how to assess the progress of my students in reading. 


□i 




□ 3 


□4 


1. I have changed my methods of teaching reading as a result of 
professional development in reading. 






□ 3 


□4 



E5. If there is anything else that you would like to tell us, please do so. 



Thank you for taking the time to answer these questions. We appreciate your willingness to describe your 
reading program. Please return your survey in the enclosed envelope to: 



Brenda Rodriguez, Senior Survey Director 
RF Implementation Study 
Abt Associates Inc. 

55 Wheeler Street 
Cambridge, MA 02138 
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Aht ID / barcode here 




0MB Number: 1875-0232 


Rf version 




Expiration Date: 10/31/07 



Reading First Implementation Study 
Principal Survey 



The U.S. Department of Education’s Policy and Program Studies Service has contracted with Aht 
Associates Inc. to conduct a national evaluation of K-3 reading instruction in Reading First and Title I 
schools. A sample of 1100 Reading First schools and a comparison group of 550 Title I schools, have 
been selected to participate in this study. The principal, reading coach, and a selected sample of K-3 
teachers from each participating school are being asked to complete a survey. Participants will help 
inform the U.S. Department of Education, Congress, policymakers, practitioners, and researchers about 
how K-3 reading instruction is implemented in schools and what strategies teachers use to provide high- 
quality, evidence-based reading instruction in grades K-3. 
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Additional Information 

The survey will take you approximately 30 minutes to complete. Although your participation in the 
survey is entirely voluntary, we strongly encourage you to participate. There are no joh-related or other 
consequences for not participating. You may also choose to answer some questions on the survey and not 
others — although we urge you to complete as many questions as possible. 

All responses to the survey will he kept confidential. All individual identifying information will he used 
only hy persons on the research team. Information such as school location (state), participants’ general 
joh titles, grades they teach, and gender will he included in the study data files to he submitted to the 
Department of Education. However, participants’ names will be stripped from all analysis data files and 
data files to be submitted to the Department of Education. We will not report any data about individual 
classrooms — all information will be reported at the grade and school levels. Neither your school nor your 
district will have access to any of the completed surveys at any time. 

Please return your completed survey to Abt Associates in the enclosed stamped, pre-addressed mailing 
envelope. If you decline to participate, simply do not return the survey in the mail. 

Thank you for your cooperation with this survey! 



According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of 
information unless such a collection displays a valid OMB control number. The valid OMB control number for this 
information collection is 1875-0232. The time required to complete this information collection is estimated to 
average 30 minutes per response, including the time to review instruction, search existing data resources, and gather 
the data needed, and complete and review the information collection. If you have any comments concerning the 
accuracy of the time estimate or suggestions for improving this form, please write to: Policy and Program Studies 
Service, U.S. Department of Education, 400 Maryland Avenue, SW, Washington, DC, 20202. 



Instructions 



Unless otherwise noted, your responses should reflect your experiences during the 2004-2005 school 
year in the school to which this survey was sent. 



■ Please complete all questions; each question includes directions for recording your answer. 



■ You are sometimes told to skip over some questions in the survey. When this happens, you will 
see an arrow with a note that tells you what question to answer like this: 

□ i Yes 

□2 No — > Skip to E4 



■ If you have any questions about how to complete the survey, please call: 

. This is a free call and will connect you with our expert interviewers 



who can assist you. 
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Background Information on You and Your School 



Al. Including this year, how many years have you been at this school in this position? 

(If less than one year, please enter ‘1’) 


Years 


Al. Including this year, what is the total number of years you have served as a principal? 

(If less than one year, please enter ‘1’) 


Years 


A3. Please provide the following information about students in your school for the current 
year (2004-2005): 




a. Total number of students currently enrolled 


Students 


b. Percentage of students who were new to the school at the beginning of this year, 
including both incoming and continuing kindergarten students (but excluding pre-K 
students) 


% 


c. Percentage of students who have left the school at any point during the year, including the 
summers between school years, excluding those who have left having completed the 
highest grade available at your school (i.e., mobility rate) 


% 


d. Average attendance rate 


% 


e. Percentage of students in your school who are English Language Learners (ELL) 


% 


A4. Please indicate the number of children currently enrolled in Grades K-3 in your 
school: 




a. Kindergarten 


Students 


b. Lirst grade 


Students 


c. Second grade 


Students 


d. Third grade 


Students 



A5. Compared to 5 years ago, has student enrollment in your school increased, decreased, or 
remained stable? 

□ i Decreased 
□2 Remained stable 
□3 Increased 

□4 Not applicable (school is new) 

A6. Please indicate the approximate percentage of students in each grade who: 



Enter % below for each grade level ... 



In 2004-2005, the percentage of students who... 


K 


1 


2 


3 


a. participate in interventions for straggling readers 


% 


% 


% 


% 










b. receive special education services 


% 


% 


% 


% 










c. receive ESL instruction 


% 


% 


% 


% 


d. receive reading instruction in a language other 
than English 


% 


% 


% 


% 










e. read at or above grade level 


% 


% 


% 


% 
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A 7 . On what information did you primarily base your estimate for question 6e? 
(Please check only one) 

□ i report cards 

□2 teacher estimates 

Q3 placement in core reading series 

Q4 state tests 

□5 other tests 

□6 informal inventories 

□7 Other (Please specify): 



A8. How many classroom teachers are assigned to grades K-3 this year (2004-05)? Please include regular 
education classes only. 



Grade Level 


Enter # of classroom teachers below... 


K 


# 


1 


# 


2 


# 


3 


# 


Other K-3 teachers (e.g., combination classrooms): 


# 



A 9 . How many special education teachers are assigned to students in grades K- 3 ? 
teachers 



AlO. Please indicate the number of classroom teachers for grades K -3 who are: 





Insert # below ... 


a. Veteran teachers with more than ten years of teaching experience 


# 


b. Experienced teachers with four to ten years of teaching experience 


# 


c. New teachers with one to three years of teaching experience 


# 


d. Total 


# 


All. Please indicate the number of classroom teachers for grades K-3 who are: 


Insert # below ... 


a. Certified teachers with elementary, reading, early childhood, or other related state 
certification 


# 


b. Teachers with probationary, provisional, or other temporary teaching certification 
(includes all teachers working towards full certification) 


# 


c. Emergency teachers without teaching certification 


# 


d. Total K-3 teachers 


# 
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A 12. Please indicate the number of classroom teachers for grades K-3 who are considered highly qualified. 

Highly qualified teachers have full state certification, at least a bachelor’s degree, and proven knowledge in the 
subject that they teach. 

Number of K-3 teachers who are highly qualified # 



A13. How many of each type of reading support personnel does your school have for grades K-3? 



Type 


Insert # below ... 


a. Certified non-classroom teachers, including special education teachers. Title 1 
teachers, and reading coaches or specialists 


# 


b. Instractional aides or assistants (during school day) 


# 


c. Tutors (before or after school) 


# 



A 14. How many certified staff positions has your school added this year (such as a reading coach) to 
support reading instruction in Grades K-3? If no positions were added, write 0. 

Number of staff positions added: # 



A15. If your school has a reading coach, from what funding source(s) is that person supported? 

(Check all that apply.) 



Reading coach is a staff member whose primary role is to provide ongoing training and 
support to classroom teachers in the delivery of effective reading instraction. This assistance 
may include planning instruction, providing demonstration lessons, observing and providing 
feedback, using assessment results to guide instruction, etc. 



□ . 

□ 2 

□ 3 

□ 4 

□ 5 



Do not have a reading coach 
Title I 

Reading First 

State/local funds 

Other (Please specify): 



A16. Did your school make “adequate yearly progress” in reading/language arts on the basis of 2003- 
04 test scores, according to NCLB accountability provisions? (Please check only one) 

“ Adequate yearly progress ” (AYP) is the amount of yearly improvement each school is expected to 
make. Each state is responsible for defining AYP and for determining the methods used to measure 
AYP. 



□ 1 Yes 

□ 2 No 

n 8 Not sure/don’t know 
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B. Resources and Support for Your School’s Reading Program 



Bl. What sources of funding are being or have been used to support your school’s reading program this year 
(2004-2005)? (Check all that apply) 



□ . 

□ 2 

□ 3 

□ 4 

□ 3 

□ 6 

□ 2 

□ s 

□ 9 

Q 10 

□ n 

□ 95 



District general funds 
State funds for reading programs 
State textbook funds 
Title I 

Title II (Professional development to improve teacher quality) 

Title III (Professional development for ELL teachers) 

Comprehensive School Reform 

2E‘ Century Community Learning Centers 

Reading First 

Professional development funds 
Private grants 

Other {Please specify): 





Enter Answer Below... 


B2. When did your school first receive funding from Reading First? 


/ (month/vear) 


B3. What is the expected duration of your Reading First grant? 


vears 


B4. When did your school first begin Reading First professional development 
activities? 


/ (month/vear) 


B5. When did your school first begin implementation of Reading First 
instructional activities in the classroom? 


/ (month/vear) 


B6. What is the amount of the Reading First grant for your school this year 
(2004-05)? 


$ .000 
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B7. Beyond financial support, has your school received external assistance this year (from district, 
state, publisher, university expert, etc.) implementing any of the following K-3 reading program 
activities? 





Check one box for each item... 




YES 


NO 


a. Selecting instructional programs/materials 


□ : 




b. Selecting assessment instruments 


□ : 




c. Selecting professional development providers 


□ : 




d. Conducting classroom observation 


□ : 




e. Conducting demonstration lessons 


□ : 




f. Interpreting assessment results 


□ : 




g. Recruiting staff with reading expertise, e.g. teachers, coaches 


□ : 




h. Setting up intervention programs for straggling readers 


□ : 




i. Planning professional development 


□ : 




j. Providing technical assistance in implementing core reading program 


□ : 




k. Providing technical assistance for using supplementary reading materials 


□ : 




1. Conducting needs assessment for professional development 


□ : 




m. Diagnosing needs of struggling readers 


□ : 




n. Reviewing reading program effectiveness 


□ : 




o. Leading teacher study groups 


□ : 





C. Reading Instructional Materials 



Cl. Which core reading program is being used to teach reading in each of Grades K-3 at this school? 



Core Reading Program is one that provides a comprehensive program of instruction on a daily 
basis in all aspects of reading. 





Please indicate the publisher, title, and publication year for the program used in each grade. 


Publisher Title Year Published 


a. Kindergarten 




b. Grade 1 




c. Grade 2 




d. Grade 3 
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C2. Do K-3 teachers use supplementary reading materials with the students to whom they teach 
reading? 

If yes , for which components of reading were the supplementary reading materials selected? 



Supplementary Reading Materials provide additional instruction in a targeted area of reading to 
all students. Do not include materials that are used only with struggling readers. Include teacher- 
made materials, if applicable. 





Check one box for each item... 
YES NO 


If YES: (Select any options that apDlv). 


Kindergarten 


J ► 


Q 1 Phonemic awareness Q 5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 


Grade 1 


J ► 


Phonemic awareness Q5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 


Grade 2 


J ► 


Q 1 Phonemic awareness Q 5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 


Grade 3 


J ► 


Phonemic awareness Q5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 
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D. Instructional Time 



Dl. How often this year is time set-aside during the school day for K-3 teachers to: 




D2. Please indicate for which grades your school has a scheduled reading block. A reading block is 
the time period that is formally scheduled for teaching reading. 

If yes , please indicate for how many minutes the reading block is scheduled. 

Does your school have a reading block in: 





Yes 


Scheduled number 
of minutes 


No 


Kindergarten 


□ i - 


— ► 1 


□ 


□ 2 


First grade 


□ i - 


— ► 1 


□ 


□ 2 


Second grade 


□ i - 


— ► 1 


1 


□ 2 


Third grade 


□ i - 


— ► 1 


LJ 


□ 2 



D3. On average, how many total minutes per day (including the reading block) are devoted this year 
to 

classroom reading instruction (not just reading activities) for students in Grades K-3? 





Please provide the number of minutes oer dav for each grade. 


a. Kindergarten 


min / dav 


b. First grade 


min / dav 


c. Second grade 


min / dav 


d. Third grade 


min / day 
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D4. For which of the following activities are state staff, district staff, the principal, and the school 

reading coach responsible? 



For each activity, check all that apply... 





State 


District 


Principal 


School’s 

reading 

coach 


N/A 


a. Selection of a specific core reading program 


□ : 




□3 


□4 


□3 


b. Selection of supplemental reading program 
materials (for use with the whole class) 


□ : 




□3 


□4 


□3 


c. Selection of intervention reading program materials 
(for use with struggling readers) 


□: 




□3 


□4 


□3 


d. Selection of reading assessment instruments 


□ : 




□3 


□4 


□ 3 



D5. This year, for which of the following activities are state staff, district staff, the principal, and the 
school reading coach responsible? 



Activities 


For each activity, check all that apply... 


School’s 

State District Principal reading 

coach 


N/A 


a. Monitoring implementation of reading program 


□, a □, a 


□3 


b. Review of teachers’ reading lesson plans 


□, a □, a 


□3 


c. Review individual students’ progress in reading 


□, a □, a 


□3 


d. Interpretation of assessment results 


□, □, □. 


□3 


e. Feedback to teachers about reading instruction 


□, a □, a 


□3 


f. Selection of reading professional development 
topics and opportunities 


□, a □, a 


□3 


D6. This school year, how often have you, as pri 

following methods? 


ncipal, evaluated K-3 reading instruction using the 

Check only one box for each item ... 




1-4 times 5-8 times ^ Once a 

v.)ricc Si 

Not at all this school this school week or 

month 

year year more 


a. Observed classroom reading instruction informally 


^3 ^4 ^3 


b. Observed classroom reading instruction using an 
evaluation form 


^3 ^4 ^3 


c. Met with teachers individually to discuss strategies 
for improving reading instruction 


□1 ^3 a ^3 


d. Met with groups of teachers to discuss strategies 
for improving reading instruction 


^3 ^4 ^3 
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D7. Has your school made any of the following changes to your reading program that took effect at 
the beginning of the current school year (2004-2005)? 





Check only one box for each item ... 


YES 


NO 


a. Adopted a new core reading program 


□ : 


□ 2 


b. Added a new intervention program for struggiing readers 


□ : 


□ 2 


c. Added new supplementary materials 


□ : 


□ 2 


d. Added new materials for ELLs 


□ : 


□ 2 


e. Adopted new reading assessments 


□ : 


□ 2 


f. Other (Piease specify): 


□: 


□2 



D8. Which of the following methods has your school used this year to assess the effectiveness of 
reading instruction for Grades K-3? (Check all that apply.) 

I 1 1 It is not assessed 

I I 2 Observation by reading coach 

I 1 3 Observation by school principal 

I 1 4 Observation by external consultant or evaluator 

I 1 5 Discussion at grade-level meetings 

I 1 6 Progress monitoring assessments 

I 1 7 Norm-referenced test data or state assessments (e.g., ITBS, SAT-9, State benchmark 
exams) 

□ 95 Other (Please specify): 
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E. Reading Interventions for Struggling Readers 



El. What methods has your school used to meet the needs of at-risk or struggling readers? For each 
method listed helow, please check whether or not you use the method at your school. 



Reading Intervention is a program designed for struggling readers to be used only with struggling 
readers in addition to the core-reading program. 



Methods of meeting needs of struggling readers 



Check only one box for 
each item ... 



Use this 
method 



Not used 





a. Use separate program materials in interventions 








b. Use core reading program with supplemental materials 






Materials 


c. Use core reading program only 








d. Use reading materials written in students’ home language 








e. Use alternative materials designed for English learners 








f. A certified reading specialist provides additional direct instmction 
to straggling readers, individually or in small groups. 








g. The classroom teacher provides additional direct instruction to 
struggling readers, individually or in small groups. 








h. The classroom teacher provides additional opportunities for 

reading skill practice for straggling readers (e.g., partner reading, 
peer tutors, audio tapes, computer programs) 








i. A certified specialist provides recommendations to classroom 
teachers on accommodations for struggling readers. (Indicate 
which type of specialist.) 




Staff 

activities 


A special education teacher 






A bilingual/ESL teacher 








Other (Please specifvl: 








j. Trained aides or volunteers work with students under the direction 
of the classroom teacher during the school day. 








k. Trained aides or volunteers work with students in a before or after 
school program. 








1. Untrained aides or volunteers work with students under the 
direction of the classroom teacher during the school day. 








m. Untrained aides or volunteers work with students in a before or 
after school program. 
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E 2 . Which of the following methods has your school used this year to identify students for reading 
interventions in Grades K- 3 ? 



Method of identifying students 


Check one box for each item... 


Use this method 


Not used 


a. Standardized achievement test scores in reading 


□ : 


□2 


b. Scores on tests that are part of the reading program 


□ : 


□2 


c. Screening test scores in reading 


□ : 


□2 


d. Diagnostic test scores in reading 


□ : 


□2 


e. Progress monitoring test scores in reading 


□ : 


□2 


f. Documented ciassroom observations 


□ : 


□2 


g. Teacher recommendations 


□ : 


□2 


h. Other schooi staff recommendations 


□ : 


□2 


i. Requests from parents 


□ : 


□2 


j. Reading coach recommendation 


□ : 


□2 


k. Other (Piease soecifv): 


□ : 


□2 



E 3 . Are reading intervention services (e.g., a reading specialist) available this year to children who 
need them? (Check ‘no’ if there are no children who need intervention services) 



□ 1 Yes 

□ 2 No 



■te 3 a. If yes, What was the average wait for reading intervention services? 
school days. (Please enter ‘zero’ if there is no wait for services) 
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F. Professional Development in Reading 



FI. During the current school year, including summer 2004, approximately how many of the 
reading professional development activities available to K-3 teachers: 

• Please choose the category that most closely descrihes your professional development. 





Check only one box for each item ... 




None 


One- 

Quarter 


One-Half 


Three- 

Quarters 


All 


a. require teachers to attend? 


□ : 




□ 3 


□ 4 


□ 3 


b. did you also attend? 


□ : 




□ 3 


□ 4 


□ 3 


c. provide teachers options among which they 
choose? 


□ : 




□ 3 


□ 4 


□ 3 


d. provide a stipend? 


□ : 




□ 3 


□ 4 


□ 3 


e. provide follow-up activities? 


□: 




□3 


□4 


□3 


f. include release time for participating teachers? 


□ : 




□ 3 


□ 4 


□ 3 


g. offer graduate college credits? 


□ : 




□ 3 


□ 4 


□ 3 


h. are held in a convenient location (e.g., activities 
held at school)? 


□: 




□3 


□4 


□3 


i. use a team-based approach (joint training of 
people who work together)? 


□: 




□3 


□4 


□3 


j . are given by trainers or facilitators who have a 
well-established reputation? 


□ : 




□ 3 


□ 4 


□ 3 
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F2. Below is a list of topics that are often covered in professional development activities that are designed for 

building administrators. 

• In column A . identify the topics that were addressed in professional development activities in which you 
participated since July of the current school year. 

• Then, in column B . please identify 5 topics in which you would like more professional development, whether 
or not your school’s professional development activities have covered these topics. 







CHECK ALL THAT 
APPLY ... 


CHECK NO MORE 
THAN 5 TOPICS ... 




Topic 


A. 

Topics addressed in 
professional 
development 


B. 

Topics in which I 
would like more 
professional 
development 




a. Phonemic Awareness 


□: 


□ 3 


Content of 
Reading 
Instruction 


b. Decoding 


□ 3 


□4 


c. Vocabulary 


□ : 


□ 3 


d. Fluency 


□3 


□4 




e. Comprehension 


□: 


□ 3 




f. How to use the core reading program 


□ 3 


□4 




g. How to use children’s literature to teach reading 


□ : 


□ 3 




h. How to use reading research to guide content of 
instruction 


□ 3 


□4 


Teaching 

Strategies 


i. How the core reading program incorporates research 
principles 


□ : 


□ 3 




j. How to use the supplemental reading program 


□3 


□4 




k. How to integrate reading and writing instruction 


□ : 


□ 3 




1. Strategies for teaching reading to ELLs 


□ 3 


□4 




m. How to evaluate a core reading program 


□ : 


□ 3 


Evaluation 


n. How to evaluate reading instruction 


□ 3 


□4 


o. How to coach teachers in reading instruction 


□ : 


□ 3 




p. How to manage reading personnel 


□ 3 


□4 




q. How to diagnose reading problems 


□: 


□ 3 


Assessment 


r. How to administer assessments 


□ 3 


□4 




s. How to interpret and use assessment data to guide 
instruction 


□ : 


□ 3 




t. How to help struggling readers with decoding 


□3 


□4 




u. How to help struggling readers with vocabulary 


□ : 


□ 3 


Struggling 

Readers 


V. How to help struggling readers with comprehension 


□ 3 


□4 


w. How to motivate readers 


□ : 


□ 3 




X. Strategies for teaching reading to students with diagnosed 
learning disabilities 


□ 3 


□4 
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F2. CONTINUED - Below is a list of topics that are often covered in professional development activities that are 
designed for building administrators. 

• In column A . identify the topics that were addressed in professional development activities in which you 
participated since July 7 ” of the current school year. 

• Then, in column B . please identify 5 topics in which you would like more professional development, whether 
or not your school’s professional development activities have covered these topics. 



(CONTINUED FROM PREVIOUS PAGE) 
Topic 


CHECK ALL 
THAT APPLY ... 


CHECK NO MORE 
THAN 5 TOPICS... 


A. 

Topics addressed in 
professional 
development 


B. 

Topics in which I’d like 
more professional 
development. 




y. How to select reading materials 




□ 3 




z. How to use content standards for curriculum planning and 
teaching 


□3 


□4 


Organization, 
management 
and support 


aa. How to select reading assessments 




□3 


bb. Alignment of reading curriculum and instruction with 
state/district assessments 


□3 


□ 4 




cc. How to work with parents 




□ 3 




dd. Classroom management 


□3 


□ 4 


Other 


ee. Other (Please specifvl: 




□3 



F3. Please indicate who participated in the completion of this questionnaire. (Check all that apply.) 
r~| 1 Principal 

r~| 2 Assistant or vice principai 

I 1 3 District reading coordinator 

I 1 5 School-hased reading coach 

r~| 6 Classroom teacher(s) 

I 1 7 School secretary 

r~| 95 Other (Piease specify): 



F4. If there is anything else that you would like to tell us about your school’ s reading program, please 
do so. 



Thank you very much for completing the survey. Please return your survey in the enclosed envelope 
addressed to: Brenda Rodriguez, Senior Survey Director, Aht Associates Inc., Attn: RF 
Implementation, 55 Wheeler Street, Cambridge, MA 02138 
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Aht ID / barcode here 




0MB Number: 1875-0232 






Expiration Date: 10/31/07 



Reading First impiementation Study 
Reading Coach Survey 



The U.S. Department of Education’ s Policy and Program Studies Service has contracted with Aht 
Associates Inc. to conduct a national evaluation of K-3 reading instruction in Reading First and Title I 
schools. A sample of 1 100 Reading First schools and a comparison group of 550 Title I schools, have 
been selected to participate in this study. The principal, reading coach, and a selected sample of K-3 
teachers from each participating school are being asked to complete a survey. Participants will help 
inform the U.S. Department of Education, Congress, policymakers, practitioners, and researchers about 
how K-3 reading instruction is implemented in schools and what strategies teachers use to provide high- 
quality, evidence-based reading instruction in grades K-3. 
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Additional Information 

The survey will take you approximately 30 minutes to complete. Although your participation in the 
survey is entirely voluntary, we strongly encourage you to participate. There are no joh-related or other 
consequences for not participating. You may also choose to answer some questions on the survey and not 
others — although we urge you to complete as many questions as possible. 

All responses to the survey will he kept confidential. All individual identifying information will he used 
only hy persons on the research team. Information such as school location (state), participants’ general 
joh titles, grades they teach, and gender will he included in the study data files to he submitted to the 
Department of Education. However, participants’ names will be stripped from all analysis data files and 
data files to be submitted to the Department of Education. We will not report any data about individual 
classrooms — all information will be reported at the grade and school levels. Neither your school nor your 
district will have access to any of the completed surveys at any time. 

Please return your completed survey to Abt Associates in the enclosed stamped, pre-addressed mailing 
envelope. If you decline to participate, simply do not return the survey in the mail. 

Thank you for your cooperation with this survey! 



According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection 
of information unless such a collection displays a valid OMB control number. The valid OMB control 
number for this information collection is 1875-0232. The time required to complete this information 
collection is estimated to average 30 minutes per response, including the time to review instruction, 
search existing data resources, gather the data needed, and complete and review the information 
collection. If you have any comments concerning the accuracy of the time estimate or suggestions for 
improving this form, please write to: Policy and Program Studies Service, U.S. Department of 
Education, 400 Maryland Avenue, SW, Washington, DC, 20202. 



Instructions 



Unless otherwise noted, your responses should reflect your experiences during the 2004-2005 school 
year in the school to which this survey was sent. 



■ Please complete all questions; each question includes directions for recording your answer. 



■ You are sometimes told to skip over some questions in the survey. When this happens, you will 
see an arrow with a note that tells you what question to answer like this: 

□ i Yes 

□2 No ^ Skip to E4 



■ If you have any questions about how to complete the survey, please call: 

. This is a free call and will connect you with our expert interviewers 



who can assist you. 
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A. 



Your Background and Experience 



Reading coach is a staff member whose primary role is to provide ongoing training and 
support to school staff in the delivery of effective reading instruction. 


Enter # below... 




Al. Including this year, for how many years have you been the K-3 reading coach for this school? 
(If less than one year, enter 1 .) 


vears 


A2. Including this year, for how many years have you worked at this school in any capacity? 
(If less than one year, enter 1 .) 


years 


A3. Including this year, how many years of classroom experience do you have, as either a teacher 
and/or reading coach? (If less than one year, enter 1). 




a. Number of years of experience 


vears 


b. Number of years of experience in grades K-3 


vears 


c. Number of years of experience at this school 


vears 


d. Number of years teaching or coaching reading 


vears 



A 4 . What is your job title? 



A 5 . Describe your certification status. Which of the following describes the teaching certificate you 
currently hold in this state? (Check one) 

Q 1 Regular or standard state certificate or advanced professional certificate 
Q2 Probationary certificate (issued after satisfying all requirements except the completion of a 
probationary period). 

□ 3 Provisional or other type of certificate given to persons who are still participating in what the 

state calls an “alternative certification program.” 

Q4 Temporary certificate (requires some additional college coursework, student teaching, and/or 
passage of a test before regular certification can be obtained). 

□ 5 Waiver or emergency certificate (issued to persons with insufficient teacher preparation who 

must complete a regular certification program in order to continue teaching). 

□5 I do not have any of the above certifications in this state. — > Skip to A6 

A 5 a. If certified, identify the area(s): (Check all that apply) 

Q 1 Elementary education 

Q2 Early childhood education 

Q3 Reading 

Q4 Special education 

Q5 Bilingual / ESOE / ESE education 

Q95 Other (Please specify): 
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A6. For what grades at this school are you currently providing coaching to staff? (Check all that 
apply.) 

I 1 1 Kindergarten 

I I 2 First grade 

I 1 3 Second grade 

I 1 4 Third grade 

I 1 5 Self-contained K -3 special education classes 

□ 95 Other (Please specify): 

B. Coach Responsibilities 





Enter number 




below... 


B 1 . This school year, for how many schools do you serve as the reading coach (including this 
school)? 


Schools 


B2. This school year, for how many teachers do you serve as the reading coach (include all 
teachers in all schools)? 


Teachers 


B3. Approximately what percentage of your time do you spend as the K-3 reading coach for 
this school? 


% 



B 4 . How central is each of the following activities to your work this year (since July 1 *‘) at this 
school? 

Please rate the activity a “ 1 ” if you do not do the activity or if it is not at all central to your role as the literacy 
coach. Rate the activity a “ 5 ” if it is absolutely central or critical to your work. 
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B5. When you coach K-3 staff, how central has each of the following activities been to your work 
this year (since July l'*^)? 

Please rate the activity a “1” if you do not do the activity or if it is not at all central to your role as the 
literacy coach. Rate the activity a “5” if it is absolutely central or critical to your work. 





Check only one box for each item... 


Coaching Activity 


Do not do 
or not at 
all central 




Somewhat 

central 




Absolutely 

central 


a. Giving demonstration lessons using core or supplemental 
materials 








□4 


□5 


b. Assisting teachers in using the core program 




□2 


□ 3 


□4 


□5 


c. Observing and providing feedback to teachers 




□2 


□ 3 


□4 


□5 


d. Assisting teachers in forming instructional groups 




□2 


□ 3 


□4 


□5 


e. Assisting teachers in designing strategies for addressing 
the needs of struggling readers 




□2 


□ 3 


□4 


□5 


f. Assisting teachers with monitoring the effectiveness of 
strategies addressing the needs of struggling readers 




□2 


°3 


□4 




g. Giving demonstrations on assessment administration and 
scoring 




□2 


□ 3 


□4 


□5 


h. Planning reading instruction with teachers 




□2 


□ 3 


□4 


□5 


i. Reviewing teachers’ lesson plans and providing feedback 




□2 


□ 3 


□4 


□5 


j. Assisting teachers in interpreting assessment results 




°2 


°3 


□4 


□5 



C. Reading Instructional Materials 



Cl. Which core reading program is being used to teach reading in each of Grades K-3 at this 
school? 



A Core Reading Program is one that provides a comprehensive program of instruction on a daily 
basis in all aspects of reading. 



Please indicate the publisher, title, and publication year for the program used in each 

grade. 





Publisher 


Title 


Year Published 


a. 








Kindergarten 








b. Grade 1 








c. Grade 2 








d. Grade 3 
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C2. Do K-3 teachers use supplementary reading materials with the students to whom they teach 
reading? 

If yes , for which components of reading were the supplementary reading materials selected? 



Supplementary Reading Materials provide additional instraction in a targeted area of reading to 
all students. Do not include materials that are used only with straggling readers. Include teacher- 
made materials, if applicable. 





Check one box for each item... 
YES NO 


If YES: (Select any options that apDlv). 


Kindergarten 


1 ► 


Phonemic awareness Q5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 


Grade 1 


□,a 

1 ► 


Phonemic awareness Q5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 


Grade 2 


□,a 

1 ► 


Phonemic awareness Q5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 


Grade 3 


□,a 

1 ► 


Phonemic awareness Q5 Comprehension 

Q2 Phonics Q7 Other 

Q3 Fluency Qg I don’t know 

Q4 Vocabulary Qo No particular area 



C3. Has your school made any of the following changes to your reading program that took effect at 
the beginning of the current school year (2004-2005)? 
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D. Instructional Time 


Dl. How often during this school year is time regularly scheduled and formally set aside during the school 

day for K-3 teachers to: 

Check only one box for each item... 




Not at all 


1-4 times 


5-8 times 


Once a 
month 


Once a 
week or 
more 


Occurs 

only 

Informally, 
as needed 


a. Collaborate on reading lesson planning and 
instruction 














b. Observe reading instruction in other 
classrooms 






□ 3 


□4 


□5 


□6 


se assessment data to plan instraction 








°4 






d. Participate in coaching with or be coached 
about reading by other teachers 




°2 


°3 


□4 






e. Be coached about reading instruction by a 
reading coach 




□ 2 


□ 3 


□4 


□5 


□6 


f. Coordinate reading interventions for struggling 
readers with special education staff 




□ 2 


□ 3 


□4 


□5 


□6 


g. Coordinate reading interventions for 
struggling readers with ELL staff 




□ 2 


□ 3 


□4 


□5 


□6 


1 Check if no ELLs 

1 ► 





D2. Please indicate for which grades your school has a scheduled reading block. If yes, please 
indicate for how many minutes the reading block is scheduled. A reading block is the time 
period that is formally scheduled for teaching reading. Does your school have a reading block in: 




D3. On average, how many total minutes per day (including the reading block) are devoted this year 
to classroom reading instruction for students in Grades K-3? 





Please provide the number of minutes per day for each 
grade... 


a. Kindergarten 


min. / dav 


b. First grade 


min. / dav 


c. Second grade 


min. / dav 


d. Third grade 


min. / dav 
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E. Reading Interventions for Struggling Readers 



El. What methods has your school used to meet the needs of at-risk or struggling readers? For each 
method listed helow, please check whether or not you use the method at your school. 



A Reading Intervention is a program designed for struggling readers to be used only with straggling 
readers in addition to the core-reading program. 







Check one for each item 


Methods for meeting needs of struggling readers 


- 


-- 


Use this 


Not used 






method 




a. Use separate program materials in interventions 








b. Use core reading program with supplemental materials 






Materials 


c. Use core reading program only 








d. Use reading materials written in students’ home language 








e. Use alternative materials designed for English learners 








f. A certified reading specialist provides additional direct instruction to 


n. 






struggling readers, individually or in small groups. 




i_i 2 




g. The classroom teacher provides additional direct instruction to 


n 


n 




struggling readers, individually or in small groups. 


' 1 


l_l2 




h. The classroom teacher provides additional opportunities for reading 








skill practice for struggling readers (e.g., partner reading, peer 
tutors, audio tapes, computer programs) 








i. A certified specialist provides recommendations to classroom 








teachers on accommodations for straggling readers. (Indicate 
which tvne of specialist.) 






Staff 

activities 


A special education teacher 






A bilingual/ESL teacher 








Other (Please specifvl: 








j. Trained aides or volunteers work with students under the direction 
of the classroom teacher during the school day. 








k. Trained aides or volunteers work with students in a before or after 
school program. 








1. Untrained aides or volunteers work with students under the direction 
of the classroom teacher during the school day. 








m. Untrained aides or volunteers work with students in a before or 
after school program. 
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F. 



Professional Development in Reading for K-3 Teachers 



FI. Below is a list of topics that are often covered in professional development activities designed to 
provide teachers with new information about the content of reading instruction. 

• In column A, identify the topics that were addressed in K-3 teachers’ professional development activities 
during the current school year, including summer 2004. 

• In column B, please identify 5 topics in which you think teachers need more professional development, 
whether or not this school’s professional development activities have covered these topics. 

• Please check all that apply in column A, and 5 choices in column B. 



Professional development is defined as any activity in which a teacher has learned about reading or reading instruction. This 
includes school-based workshops, meetings with reading coaches, and meetings with a study group of other teachers. 







CHECK ALL 
THAT APPLY 


INDICATE NO 
MORE THAN 5 
TOPICS 




Topic 


A. 

Topics addressed 
in professional 
development for 
teachers 


B. 

Topics in which 
teachers need 
more 

professional 

development 


Phonemic 

Awareness 


a. Building phonological awareness, e.g. rhymes, dividing 
spoken language into sentences, words, syllables 


□ . 


□ 3 


b. Identifying, adding, deleting sounds in spoken words 


□ 3 


□ 4 




c. Blending phonemes to form words 


□ . 


□ 3 




e. Teaching letter-sound correspondence 


□ 3 


□ 4 


Decoding 


f. Teaching letter patterns (blends, digraphs, diphthongs) 


□ . 


□ 3 


g. Using syllable patterns to read words 


□ 3 


□ 4 




h. Teaching component parts: roots, prefixes, suffixes 


□ . 


□ 3 




j. Teaching use of dictionary, thesaurus 


□ 3 


□ 4 


Vocabulary 


k. Direct teaching of vocabulary words and their meaning 


□ . 


□ 3 




1. Antonyms and synonyms 


□ 3 


□ 4 




n. Teaching sight words 


□ . 


□ 3 


Fluency 


o. Guided oral reading 


□ 3 


□ 4 




p. Encouraging expression while reading 


□ . 


□ 3 




r. Setting motivation/asking prediction/preview questions 


□ 3 


□ 4 




s. Constracting information about character, setting, and main 
events 


□ . 


□ 3 




t. Summarizing main ideas in narrative and informational text 


□ 3 


□ 4 


Comprehension 


u. Self-monitoring strategies 




□ 3 




v. Asking questions at different levels (literal, inferential) 




□ 3 




w. Strategies for organizing text structure, e.g. story maps 


□ 3 


□ 4 




X. Shared book conversations 


□ . 


□ 3 


Other 


y. Other topic in the 5 dimensions of reading. (Please specify:) 


□ 3 


□ 4 
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F2. Below is a list of topics that are often covered in professional development activities that are 
designed to provide teachers with new information about teaching strategies used during 
reading instruction. 

• In column A, identify the topics that were addressed in K-3 teachers’ professional development 
activities during the current school year, including summer 2004. 

• Then, in column B, please identify 5 topics in which teachers need more professional development, 
whether or not this school’s professional development activities have covered these topics. 

• Please check all that apply in column A, and 5 choices in column B. 



Topic 


CHECK ALL 
THAT APPLY 


INDICATE NO MORE 
THAN 5 TOPICS 


A. 

Topics addressed in 
professional 
development for 
teachers 


B. 

Topics in which 
teachers need more 
professional 
development 




a. How to use the core reading program 




□ 3 




b. How to use children’s literature to teach reading 


□ 3 


□ 4 


Teaching 

Strategies 


c. How to use reading research to guide content of 
instraction 




□ 3 


d. How the core reading program incorporates research 
principles 


□ 3 


□ 4 




e. How to use the supplemental reading program(s) 




□ 3 




f. How to integrate reading and writing instruction 


□ 3 


□ 4 




g. Strategies for teaching reading to ELLs 




□ 3 


Grouping 


h. Learning styles 


□ 3 


□ 4 


i. How to organize small group instruction 




□ 3 




j . How to diagnose reading problems 


□ 3 


□ 4 


Assessment 


k. How to administer assessments 




□ 3 




1. How to interpret and use assessment data to guide 
instruction 


□ 3 


□ 4 




m. How to help struggling readers with decoding 


□ . 


□ 3 




n. How to help struggling readers with vocabulary 


□ 3 


□ 4 


Struggiing 

Readers 


o. How to help struggling readers with comprehension 


□ > 


□ 3 


p. How to motivate readers 


□ 3 


□ 4 




q. Strategies for teaching reading to students with 
diagnosed learning disabilities 




□ 3 




r. How to use state/district content standards for curriculum 
planning and teaching 


□ 3 


□ 4 


Organization/ 

pianning 


s. How to align reading curriculum and instmction with 
state/district assessments 




□ 3 


t. How to work with parents 


□ 3 


□ 4 




u. Classroom management 




□ 3 


Other 


v. Other (Please specifvl: 


□ 3 


□ 4 
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F3. During the current school year, including summer 2004, approximately how many of the reading 
professional development activities available to K -3 teachers: (Please choose the category that 
most closely descrihes K-3 teachers’ professional development.) 





Check only one box for each item ... 




None 


One- 


One 


Three- 


All 




Quarter 


Half 


Quarters 


a. Require teachers to attend? 




□2 


□ 3 


□4 


□5 


b. Were attended by the principal? 




°2 




□4 


□5 


c. Provide teachers options among which they can choose? 




□2 


□ 3 


□4 




d. Provide a stipend? 




°2 




°4 




e. Provide follow-up activities? 




□ 2 


□ 3 


□4 


□5 


f. Include release time for participating teachers? 




□ 2 


□ 3 


□4 


□5 


g. Offer graduate college credits? 




□ 2 


□ 3 


□4 




h. Are held in a convenient location (e.g. at school)? 




□ 2 


□ 3 


□4 


□5 


i. Use a team-based approach (joint training of people who 
work together)? 




□ 2 


□ 3 


□4 


□5 


j . Are given by trainers or facilitators who have a well- 
established reputation? 




□ 2 


□ 3 


□4 


□5 
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G. Professional Development for Reading Coaches 



Gl. Below is a list of professional development topics for reading coaches in which you may have 
participated. 

• In column A, identify any topics that were addressed in reading coaches’ professional development 
activities during the current school year, including summer 2004. 

• Then, in column B, please identify no more than 3 topics in which you would like more professional 



development, whether or not this school’s professional development activities have covered these topics. 
• Please check all that apply in column A, and 3 choices in column B. 





CHECK ALL THAT 
APPLY 


CHECK NO MORE 
THAN 3 TOPICS 


Topics 


A. Topics addressed in 
professional 
development for 
reading coaches 


B. Topics in which 
you’d like more 
professional 
development 


a. How to use reading assessment data to guide instmction. 




□ 3 


b. What are the types of assessments: screening, diagnostic, progress 
monitoring, and outcome. 


□ 3 


□ 4 


c. How to use assessment data to form instractional groups. 




□ 3 


d. How to provide constructive feedback to teachers. 


□ 3 


□ 4 


e. How to establish credibility with teachers. 


□ . 


□ 3 


f. Essential components of scientifically based reading instruction. 


□ 3 


□ 4 


g. What is the role of the reading coach in fostering change. 




□ 3 


h. How to plan instructional interventions for struggling students. 


□ 3 


□ 4 


i. Classroom management within the literacy block time. 




□ 3 


j. How to conduct effective grade level meetings. 


□ 3 


□ 4 


k. How to help teachers identify appropriate instructional materials. 




□ 3 


1. How to help teachers make reading instruction systematic and 
explicit. 


□ 3 


□ 4 


m. How to conduct demonstration lessons. 


□ . 


□ 3 


n. How to conduct classroom observations. 


□ 3 


□ 4 


o. How to provide onsite professional development. 


□ . 


□ 3 


p. Other (Please specify): 


□ 3 


□ 4 
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H. Reading Instruction 



HI. This item asks you to describe your school using the statements below. Please read each 

statement, and indicate whether the statement is a good description of your school on a scale from 
a “Very inaccurate” description of your school to a “Very accurate” description of your school. 



Check one box for each item... 



In this school ... 


Vprv 








Vprv 




y ciy 

inaccurate 






r 


y ciy 

accurate 


a. K-3 teachers are knowledgeable about scientifically based reading 
instruction. 




□2 


□ 3 


□4 




b. K-3 teachers are motivated to improve reading instruction. 




□2 


□ 3 


□4 


□5 


c. Reading instmction in K-3 classrooms is aligned with the state 
reading/language arts content standards. 




□2 


□ 3 


□4 


□5 


d. There is a school-wide focus on reading and language arts. 




□2 


□ 3 


□4 


□5 


e. K-3 teachers are experienced with the core reading program. 




□2 


□ 3 


□4 


□5 


f. K-3 teachers are experienced with supplemental reading materials. 




°2 


□3 


°4 




g. K-3 teachers are experienced with reading intervention materials and 
strategies. 




°2 




°4 




h. K-3 classrooms have ample, high quality instructional materials. 




□2 


□ 3 


□4 


□5 


i. Teachers use a variety of instractional materials to fill in gaps in the 
core program. 




□2 


□ 3 


□4 




j. The core reading program is aligned with scientifically based reading 
research. 




□2 


□ 3 


□4 


□5 


k. Supplemental reading materials are aligned with scientifically based 
reading research. 




°2 


^3 


°4 




1. Reading intervention materials are aligned with scientifically based 
reading research. 




□2 


□ 3 


□4 


□5 


m. The reading coach has the support of the school principal. 




□2 


□ 3 


□4 




n. K-3 teachers seek the assistance of the reading coach to improve their 
reading instruction. 




□2 


□ 3 


□4 


□5 


o. Sufficient time during the school day is allotted for reading instruction. 




□2 


□ 3 


□4 




p. Sufficient time during the school day is allotted for teacher planning. 




□2 


□ 3 


□4 


□5 


q. K-3 teachers collaborate and plan for reading instruction. 




□2 


□ 3 


□4 


□5 


r. Sufficient time during the school day is allotted for professional 
development. 




□2 


□ 3 


□4 


□5 


s. Sufficient time during the school day is allotted for professional 
development. 




°2 


^3 


°4 




t. Reading assessments are used to screen students for reading difficulties. 




□2 


□ 3 


□4 


□5 


u. Diagnostic assessments are used to identify strengths and weaknesses 
of struggling readers. 




□2 


□ 3 


□4 


□5 


V. Reading assessments are used to monitor student progress. 




□2 


□ 3 


□4 


□5 


X. Assessment data are used to group students for instruction. 




°2 


^3 


°4 




y. Assessment data are used to guide and/or modify instruction. 




□2 


□ 3 


□4 


□5 


z. The district provides direction concerning reading instruction. 




□2 


□ 3 


□4 


□5 


aa. The state provides direction concerning reading instruction. 




□2 


□ 3 


□4 


□5 


bb. K-3 teachers make an effort to involve parents in their children’s 
reading instruction. 




□2 


□ 3 


□4 


□5 
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I. 



Conclusions and Recommendations 



II. What are the most significant changes this school made this year in K-3 reading instruction? 



12. What are the most significant obstacles you encountered this year in your efforts to work with this 
school to improve reading instruction? 



13. If there is anything else that you would like to tell us, please do so. 



Thank you for taking the time to answer these questions. We appreciate your willingness to describe your 
reading program. Please return your survey in the enclosed envelope to: 



Brenda Rodriguez, Senior Survey Director 
RF Implementation Study 
Abt Associates Inc. 

55 Wheeler Street 
Cambridge, MA 02138 
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Exhibit C.2.a 

Percent of Staff that Are in Their Current Reading First and Titie i Schoois for Three Years or 
Less, 2004-05 Schooi Year 



Percent 




1st year in this 2-3 years in 1st year in this 2-3 years in 1st year in this 2-3 years in 

school this schooi school this school school this school 

Principals Teachers Reading Coaches 

Source: Principal Reading Coach and Teacher Surveys, Question Al. 



Exhibit C.2.C 

Sources of Funding for K-3 Reading Programs in the Total Population of Reading First 
and Title 1 Schools, 2004-05 School Year 




Reading First Schools 


Title 1 Schools 




Source of Funding 


Percent 


Percent 


p-value 


Title 1 


91% 


97% 


.000 


District general funds 


79 


87 


.000 


State funds for reading programs 


49 


52 


.334 


State textbook funds 


41 


48 


.014 


Professional development funds 


43 


48 


.038 


Title II 


38 


42 


.134 


Title III 


24 


21 


.153 


21st Century Learning Centers 


17 


14 


.129 


Comprehensive school reform 


10 


10 


.992 


Private grants 


8 


15 


.000 


Other 


10 


11 


.576 


School Size (all schools) 


5.1 sources 


4.5 sources 


.000 


... 1 -249 students 


4.6 sources 


4.3 sources 


.190 


...250-499 students 


4.9 sources 


4.4 sources 


.001 


...500-749 students 


5.3 sources 


4.4 sources 


.000 


...750+ students 


5.4 sources 


4.9 sources 


.086 


Source: Principal Survey, Question Bl. 
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Exhibit C.3.5 



Responsibiiity for Seiection of Reading Materiais in Mature Reading First and Titie i Schoois, 

2004-05 Schooi Year 

Mature Reading First Title I 

Standard Standard 





Percent 


Error 


Percent 


Error 


p-value 


Selection of a specific core reading program 

State 


30% 


2.1 


18% 


1.6 


.000 


District 


88 


1.5 


83 


1.6 


.012 


Principal 


33 


2.2 


38 


2.1 


.096 


School's reading coach 


26 


2.1 


18 


1.6 


.001 


Selection of supplemental reading program 
materials 

State 


18 


1.8 


6 


1.0 


.000 


District 


61 


2.3 


56 


2.2 


.071 


Principal 


66 


2.3 


68 


2.1 


.458 


School's reading coach 


66 


2.2 


40 


2.2 


.000 


Selection of intervention reading program 
materials (for use with struggling readers) 

State 


18 


1.8 


5 


0.9 


.000 


District 


64 


2.3 


56 


2.2 


.012 


Principal 


62 


2.3 


70 


2.0 


.017 


School's reading coach 


67 


2.2 


43 


2.2 


.000 



Source: Principal Survey, Question D4. 
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Exhibit C.3.6.a 



Core Reading Programs Used by iess than 1 Percent of Reading First Schools, 2004-05 School Year 



Reading First Schools 


Percentage of Schools in Sample that 
Use the Program 


Publisher 


Program 


< 1% 


Benchmark 


Benchmark phonetics 




Caron-Dellosa 


Unspecified 




Hampton Brown 


Phonics and Friends 




Heinemann 


Fountas Pinnel units of study 




Houghton Mifflin 


Invitation to Literacy 




John Hopkins 


Unspecified 




McGraw-Hill 


Breakthrough to Literacy 
Spotlight on Literacy 




Mondo 


Mondo Book Shop 




Saxon 


Saxon Phonics 




Scholastic 


Guided Reading 
Literacy Place 




Scott Foresman 


Celebrate Reading 
Lectura 




Waterford 


Waterford Early Reading 




Wright Group 


Unspecified 


Source: Reading Coach Survey, Principal Survey Question Cl. 




Exhibit C.3.6.b 






Core Reading Programs Used by iess than 1 Percent of Title I Schools, 2004-05 School Year 


Title I Schools 


Percentage of Schools in Sample that 
Use the Program 


Publisher 


Program 


< 1% 


Addison Wesley 


Unspecified 




America's Choice 


America's Choice 




Benchmark 


Benchmark phonetics 




Hampton Brown 


Phonics and Friends 




Houghton Mifflin 


Lectura 

Soar 2 Success 
Literacy Experience 




Language Circle 


Project Read 




Literacy First 


Unspecified 




McGraw-Hill 


New View 
Other 

Breakthrough to Literacy 




Metro 


Early Reading 




Owens 


Unspecified 




Pearson Learning Group Unspecified 

Sing, Spell, Read & Write 




Scholastic 


Guided Reading 


Source: Reading Coach Survey, Principal Survey Question Cl. 
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Exhibit C.3.7 

Materiais Used with Engiish Language Learners (ELLs) in Mature Reading First and Titie i 
Schoois, 2004-05 Schooi Year 



Reading First Title I 

Teachers Teachers 







Standard 




Standard 






Percent 


Error 


Percent 


Error 


p-vaiue 


Do not teach ELLs 

Kindergarten 


36% 


.03 


53% 


.03 


.000 


1st grade 


34 


.02 


52 


.03 


.000 


2nd grade 


34 


.02 


51 


.03 


.000 


3rd grade 

Among those who teach ELLs: 


39 


.03 


54 


.03 


.000 


Core reading program in native language 

Kindergarten 


10 


.02 


14 


.03 


.254 


1st grade 


10 


.02 


15 


.03 


.179 


2nd grade 


15 


.03 


12 


.03 


.513 


3rd grade 


12 


.03 


9 


.02 


.427 


ELL students use same materiais as other 
students 

Kindergarten 


78 


.03 


84 


.03 


.157 


1st grade 


72 


.03 


75 


.03 


.513 


2nd grade 


74 


.03 


70 


.04 


.517 


3rd grade 


72 


.03 


74 


.03 


.560 


Core and supplemental materials in native 
ianguage 

Kindergarten 


10 


.02 


16 


.03 


.145 


1st grade 


12 


.02 


14 


.03 


.494 


2nd grade 


15 


.03 


12 


.02 


.506 


3rd grade 


21 


.03 


16 


.03 


.264 


Core and supplemental materials in Engiish 
especiaily for ELLs 

Kindergarten 


57 


.04 


30 


.04 


.000 


1st grade 


55 


.04 


42 


.04 


.013 


2nd grade 


61 


.04 


45 


.04 


.002 


3rd grade 


66 


.04 


46 


.04 


.000 


Alternative core materiais in English geared 
toward the instructionai ievel of ELLs 

Kindergarten 


23 


.03 


19 


.03 


.346 


1st grade 


27 


.03 


26 


.03 


.798 


2nd grade 


25 


.03 


23 


.03 


.639 


3rd grade 


33 


.04 


30 


.04 


.479 



Source: Teacher Survey, Question C8. 
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Exhibit C.4.2 



Teachers’ Use of Supports Used to Meet the Needs of Struggiing Readers in RF and Titie i 
Schoois, 2004-05 Schooi Year 



Type of Support 


Grade 


Teacher 


p- value 


Reading First 


Title 1 


Percent 


Standard. 

Error 


Percent 


Standard 

Error 


Diagnostic assessment to 


K 


75% 


0.025 


59% 


0.025 


0.000 


determine core deficits 


1 


78 


0.025 


69 


0.023 


0.006 




2 


72 


0.026 


65 


0.025 


0.082 




3 


70 


0.027 


61 


0.025 


0.023 


Extra practice in the 


K 


98 


0.006 


97 


0.007 


0.122 


ciassroom with phonemic 


1 


95 


0.013 


93 


0.012 


0.386 


awareness 


2 


92 


0.015 


89 


0.016 


0.155 




3 


85 


0.019 


74 


0.022 


0.000 


Extra practice in the 


K 


97 


0.008 


92 


0.013 


0.004 


ciassroom with decoding 


1 


97 


0.010 


95 


0.011 


0.241 




2 


95 


0.011 


93 


0.012 


0.192 




3 


92 


0.014 


83 


0.019 


0.000 


Extra practice in the 


K 


80 


0.023 


78 


0.021 


0.529 


ciassroom with fiuency 


1 


92 


0.016 


90 


0.015 


0.386 




2 


96 


0.011 


93 


0.013 


0.098 




3 


97 


0.010 


90 


0.015 


0.000 


Extra practice in the 


K 


83 


0.021 


85 


0.017 


0.584 


ciassroom with 


1 


89 


0.019 


91 


0.014 


0.392 


comprehension 


2 


90 


0.017 


92 


0.014 


0.478 




3 


93 


0.016 


94 


0.011 


0.772 


Extra instructionai time 


K 


90 


0.017 


86 


0.017 


0.107 




1 


86 


0.021 


84 


0.018 


0.356 




2 


84 


0.022 


85 


0.018 


0.591 




3 


83 


0.022 


83 


0.019 


0.874 


Piacement in materiais that 


K 


70 


0.025 


62 


0.025 


0.025 


supplement core reading 


1 


73 


0.025 


73 


0.022 


0.968 


program 


2 


72 


0.026 


68 


0.024 


0.195 




3 


74 


0.025 


66 


0.024 


0.023 


Placement in different level 


K 


44 


0.027 


48 


0.026 


0.278 


of core reading program 


1 


51 


0.027 


56 


0.025 


0.207 




2 


47 


0.028 


49 


0.026 


0.585 




3 


46 


0.029 


44 


0.026 


0.530 


Placement in separate core 


K 


19 


0.021 


24 


0.022 


0.124 


reading program 


1 


23 


0.023 


25 


0.023 


0.375 




2 


23 


0.023 


26 


0.022 


0.332 




3 


22 


0.024 


31 


0.025 


0.013 


Placement in special 


K 


54 


0.027 


45 


0.026 


0.013 


intervention program 


1 


66 


0.027 


65 


0.024 


0.877 




2 


70 


0.026 


61 


0.025 


0.008 




3 


68 


0.027 


60 


0.025 


0.046 


Work with tutor on one-to- 


K 


59 


0.027 


64 


0.024 


0.209 


one basis 


1 


60 


0.028 


63 


0.024 


0.376 




2 


57 


0.028 


66 


0.024 


0.028 




3 


54 


0.028 


60 


0.025 


0.092 


Work with reading 


K 


20 


0.022 


17 


0.020 


0.289 


specialist on one-to-one 


1 


27 


0.024 


37 


0.025 


0.004 


basis 


2 


30 


0.025 


37 


0.026 


0.063 




3 


33 


0.026 


36 


0.025 


0.492 


Work with reading 


K 


34 


0.026 


31 


0.024 


0.459 


specialist in small group 


1 


48 


0.027 


53 


0.026 


0.178 




2 


48 


0.028 


53 


0.026 


0.183 




3 


51 


0.028 


58 


0.026 


0.102 


Work with more advanced 


K 


79 


0.023 


74 


0.021 


0.178 


peer 


1 


76 


0.024 


77 


0.021 


0.829 
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Exhibit C.4.2 



Teachers’ Use of Supports Used to Meet the Needs of Struggiing Readers in RF and Titie i 
Schoois, 2004-05 Schooi Year 



Type of Support 


Grade 


Teacher 


p- value 


Reading First 


Title 1 


Percent 


Standard. 

Error 


Percent 


Standard 

Error 




2 


83 


0.022 


77 


0.021 


0.091 




3 


81 


0.022 


73 


0.024 


0.017 


Special materials for 


K 


71 


0.025 


75 


0.021 


0.189 


parents to provide practice 


1 


69 


0.025 


67 


0.024 


0.591 




2 


59 


0.028 


56 


0.026 


0.396 




3 


47 


0.029 


52 


0.026 


0.255 


If English language 


K 


74 


0.032 


73 


0.035 


0.792 


learner(s), ESL instruction 


1 


79 


0.029 


76 


0.031 


0.474 




2 


75 


0.032 


73 


0.033 


0.715 




3 


74 


0.034 


73 


0.033 


0.794 


If English language 


K 


27 


0.032 


32 


0.038 


0.276 


learner(s), provide reading 


1 


24 


0.031 


24 


0.032 


0.905 


instruction in home 


2 


20 


0.030 


27 


0.033 


0.112 


language 


3 


17 


0.028 


22 


0.034 


0.214 


If English language 


K 


61 


0.036 


53 


0.039 


0.143 


learner(s), in classroom 


1 


66 


0.034 


55 


0.038 


0.031 


help in reading from ELL 


2 


66 


0.035 


55 


0.037 


0.023 


teaoher 


3 


63 


0.038 


57 


0.038 


0.331 



Source: Teacher Survey, Question (C9. 
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Exhibit C.4.6 



Amount of Time Teachers in Reading First and Titie i Schoois Set Aside to Coordinate 
interventions with Staff, 2004-05 Schooi Year 









Teacher 










Reading First 


Title 1 




Coordination with: 


Grade 


Percent 


Standard 

Error 


Percent 


Standard 

Error 


p-vaiue 


Special Education staff 














Not at all 


K 


51% 


0.028 


44% 


0.025 


0.045 




1 


42 


0.028 


34 


0.024 


0.055 




2 


36 


0.027 


30 


0.023 


0.007 




3 


32 


0.026 


34 


0.025 


0.615 


Infrequently (monthly or less) 


K 


17 


0.020 


26 


0.022 


0.126 




1 


27 


0.025 


28 


0.023 


0.593 




2 


28 


0.025 


33 


0.024 


0.126 




3 


32 


0.027 


32 


0.025 


0.953 


Weekly 


K 


9 


0.016 


7 


0.013 


0.405 




1 


11 


0.017 


9 


0.014 


0.353 




2 


11 


0.016 


9 


0.015 


0.541 




3 


13 


0.019 


15 


0.017 


0.350 


Informally as needed 


K 


22 


0.023 


23 


0.022 


0.727 




1 


20 


0.022 


28 


0.022 


0.020 




2 


25 


0.025 


28 


0.024 


0.398 




3 


23 


0.024 


19 


0.020 


0.162 


ELL staff 














Not at all 


K 


45 


0.037 


39 


0.037 


0.245 




1 


34 


0.035 


31 


0.035 


0.521 




2 


34 


0.035 


31 


0.036 


0.462 




3 


33 


0.037 


33 


0.038 


0.931 


Infrequently (monthly or less) 


K 


24 


0.032 


28 


0.037 


0.433 




1 


34 


0.035 


28 


0.033 


0.171 




2 


32 


0.035 


29 


0.035 


0.540 




3 


30 


0.036 


35 


0.038 


0.318 


Weekly 


K 


13 


0.023 


9 


0.022 


0.224 




1 


11 


0.021 


14 


0.026 


0.480 




2 


12 


0.024 


15 


0.029 


0.443 




3 


12 


0.024 


10 


0.021 


0.612 


Informally as needed 


K 


18 


0.029 


24 


0.034 


0.167 




1 


20 


0.027 


27 


0.034 


0.088 




2 


21 


0.029 


25 


0.033 


0.379 




3 


25 


0.032 


22 


0.033 


0.501 



Source: Teacher Survey, Questions C3F and C3G. 
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Exhibit C.5.1 



Assistance for K-3 Reading Assessment Activities in Mature Reading First and Titie i 
Schoois, 2004-05 Schooi Year 





Reading First 
Teachers 


Titie 1 Teachers 




Assessment Activity 


Standard 

Percent Error 


Percent 


Standard 

Error 


p-vaiue 


Selecting assessment instruments 


76% 


2.14 


56% 


2.22 


.000 


Interpreting assessment results 


82 


1.89 


70 


2.02 


.000 



Source: Principal Survey, Question B2. 



Exhibit C.5.2 








Responsibiiity for Reading Assessment Activities in Mature Reading First and Titie i Schoois, 
2004-05 Schooi Year 


Assessment Activity 

Person(s) Responsible 


Reading First Teachers 


Titie 1 Teachers 




Standard 

Percent Error 


Standard 

Percent Error 


p-value 



Selecting assessment instruments 


State staff 


51% 


2.41 


30% 


2.01 


.000 


District staff 


78 


2.00 


80 


1.75 


.472 


Principal 


40 


2.36 


52 


2.22 


.000 


School’s reading coach 


38 


2.33 


32 


2.06 


.044 


Interpreting assessment results 


State staff 


24 


2.07 


13 


1.51 


.000 


District staff 


56 


2.43 


57 


2.20 


.697 


Principal 


89 


1.48 


90 


1.34 


.817 


School’s reading coach 


92 


1.28 


51 


2.16 


.000 


Source: Principal Survey, Questions D4 and D5. 
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Exhibit C.5.3a 

Types of Assessments (Formai and informai) Reported as Usefui by Mature Reading First and 
Titie i Kindergarten Teachers, by Assessment Purpose, 2004-05 Schooi Year 



Assessment Purpose 


Reading First 
Teachers 


Title 1 Teachers 




Type of Assessment 




Standard 




Standard 




Percent 


Error 


Percent 


Error 


p-value 


Placing or grouping of students 


Formal assessments 












Core or supplementary program assessment 


34 


2.76 


23 


2.29 


.003 


District assessment 


9 


1.66 


5 


1.21 


.094 


Standardized assessment 


48 


2.75 


40 


2.70 


.053 


State-specific assessment 


17 


1.95 


16 


2.02 


.961 


Informal assessments 


33 


2.75 


40 


2.68 


.055 


Not able to categorize 


38 


2.87 


46 


2.75 


.039 


Determining student mastery of skills 


Formal assessments 












Core or supplementary program assessment 


42 


2.86 


27 


2.44 


.000 


District assessment 


10 


1.85 


6 


1.24 


.057 


Standardized assessment 


39 


2.54 


37 


2.62 


.471 


State-specific assessment 


13 


1.81 


16 


1.94 


.359 


Informal assessments 


34 


2.78 


46 


2.67 


.003 


Not able to categorize 


39 


2.88 


44 


2.68 


.202 


Identifying the core deficits of struggling students 


Formal assessments 












Core or supplementary program assessment 


33 


2.79 


22 


2.33 


.001 


District assessment 


10 


1.89 


5 


1.22 


.034 


Standardized assessment 


46 


2.70 


37 


2.64 


.019 


State-specific assessment 


14 


1.83 


15 


1.99 


.597 


Informal assessments 


29 


2.69 


45 


2.69 


.000 


Not able to categorize 


36 


2.88 


41 


2.74 


.181 



Source: Teacher Survey, Question C7. 

“ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three 



assessments per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment 
was counted only once (e.g., if the DIB ELS Nonsense Word Fluency test and the DIB ELS Phonemic Segmentation Fluency 
test were used to place or group students, then the DIBELS was counted only once for that purpose). 

We were not able to categorize 35^0 percent of teachers’ responses. “Not able to categorize” includes responses that were 
too vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, 
skill test, fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); 
and responses that were not discemable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language 
and Literacy Assessment). 

In order to be grouped as a “district assessment,” the teacher had to clearly label it as a district test. 
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Exhibit C.5.3b 



Types of Assessments (Formal and Informal) Reported as Useful by Mature Reading First and 
Titie i fsf-grade Teachers, by Assessment Purpose, 2004-05 Schooi Year 

Reading First 

Teachers Title I Teachers 

Standard Standard 

Percent Error Percent Error p-value 



Assessment Purpose 

Type of Assessment 



Placing or grouping of students 



Formal assessments 












Core or supplementary program assessment 


35 


2.84 


26 


2.21 


.008 


District assessment 


3 


1.06 


3 


0.89 


.775 


Standardized assessment 


52 


2.72 


49 


2.65 


.399 


State-specific assessment 


14 


1.81 


15 


1.79 


.763 


Informal assessments 


29 


2.70 


41 


2.61 


.001 


Not able to categorize 


34 


2.78 


37 


2.52 


.488 


Determining student mastery of skills 


Formal assessments 












Core or supplementary program assessment 


54 


2.92 


42 


2.56 


.003 


District assessment 


2 


0.83 


2 


0.59 


.501 


Standardized assessment 


38 


2.62 


34 


2.59 


.362 


State-specific assessment 


13 


1.77 


12 


1.64 


.841 


Informal assessments 


28 


2.63 


39 


2.62 


.002 


Not able to categorize 


39 


2.93 


41 


2.57 


.532 



Identifying the core deficits of struggling students 



Formal assessments 



Core or supplementary program assessment 


41 


2.94 


29 


2.45 


.002 


District assessment 


1 


0.61 


1 


0.58 


.901 


Standardized assessment 


45 


2.59 


41 


2.68 


.247 


State-specific assessment 


15 


1.91 


17 


1.84 


.650 


Informal assessments 


24 


2.50 


43 


2.70 


.000 


Not able to categorize 


36 


2.93 


39 


2.57 


.403 



Source: Teacher Survey, Question C7. 

““ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three 
assessments per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment 
was counted only once (e.g., if the DIB ELS Nonsense Word Fluency test and the DIB ELS Phonemic Segmentation Fluency 
test were used to place or group students, then the DIBELS was counted only once for that purpose). 

’’ We were not able to categorize 35^0 percent of teachers’ responses. “Not able to categorize” includes responses that were 
too vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, 
skill test, fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); 
and responses that were not discemable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language 
and Literacy Assessment). 

In order to be grouped as a “district assessment,” the teacher had to clearly label it as a district test. 
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Exhibit C.5.3c 

Types of Assessments (Formai and informai) Reported as Usefui by Mature Reading First and 
Titie i 2/7d-grade Teachers, by Assessment Purpose, 2004-05 Schooi Year 

Reading First Title I 



Assessment Purpose Standard Standard 



Type of Assessment Percent 


Error 


Percent 


Error 


p-value 


Placing or grouping of students 


Formal assessments 












Core or supplementary program assessment 


40% 


2.85 


33% 


2.52 


.074 


District assessment 


3 


0.98 


5 


1.18 


.144 


Standardized assessment 


53 


2.68 


49 


2.72 


.338 


State-specific assessment 


16 


1.93 


11 


1.61 


.078 


Informal assessments 


25 


2.51 


36 


2.63 


.002 


Not able to categorize 


35 


2.76 


35 


2.64 


.944 


Determining student mastery of skills 


Formal assessments 












Core or supplementary program assessment 


55 


2.92 


46 


2.67 


.034 


District assessment 


4 


1.13 


4 


1.01 


.985 


Standardized assessment 


30 


2.51 


30 


2.51 


.903 


State-specific assessment 


14 


1.80 


9 


1.50 


.016 


Informal assessments 


28 


2.65 


35 


2.64 


.057 


Not able to categorize 


39 


2.92 


46 


2.72 


.075 


Identifying the core deficits of struggling students 


Formal assessments 












Core or supplementary program assessment 


42 


2.92 


30 


2.53 


.003 


District assessment 


2 


0.77 


3 


0.97 


.266 


Standardized assessment 


40 


2.70 


36 


2.67 


.311 


State-specific assessment 


15 


1.82 


15 


1.94 


.956 


Informal assessments 


27 


2.65 


37 


2.77 


.006 


Not able to categorize 


39 


2.86 


41 


2.81 


.689 



Source: Teacher Survey, Question C7. 

“ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three 



assessments per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment 
was counted only once (e.g., if the DIB ELS Nonsense Word Fluency test and the DIB ELS Phonemic Segmentation Fluency 
test were used to place or group students, then the DIBELS was counted only once for that purpose). 

We were not able to categorize 35^0 percent of teachers’ responses. “Not able to categorize” includes responses that were 
too vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, 
skill test, fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); 
and responses that were not discemable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language 
and Literacy Assessment). 

In order to be grouped as a “district assessment,” the teacher had to clearly label it as a district test. 
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Exhibit C.5.3d 

Types of Assessments (Formai and informai) Reported as Usefui by Mature Reading First and 
Titie i fade Teachers, by Assessment Purpose, 2004-05 Schooi Year^'^ 



Assessment Purpose 

Type of Assessment 


Reading First Teachers 
Standard 

Percent Error 


Title 1 Teachers 
Standard 

Percent Error 


p-value 


Placing or grouping of students 


Formal assessments 












Core or supplementary program assessment 


33% 


2.81 


27% 


2.33 


.070 


District assessment 


4 


1.14 


6 


1.29 


.326 


Standardized assessment 


48 


2.70 


41 


2.61 


.076 


State-specific assessment 


13 


1.92 


11 


1.63 


.403 


Informal assessments 


24 


2.55 


28 


2.55 


.208 


Not able to categorize 


37 


2.93 


45 


2.69 


.062 


Determining student mastery of skills 


Formal assessments 












Core or supplementary program assessment 


52 


3.00 


41 


2.69 


.003 


District assessment 


4 


1.16 


6 


1.32 


.367 


Standardized assessment 


28 


2.45 


23 


2.34 


.175 


State-specific assessment 


16 


2.12 


13 


1.77 


.315 


Informal assessments 


26 


2.65 


37 


2.62 


.004 


Not able to categorize 


39 


2.96 


49 


2.79 


.018 


Identifying the core deficits of struggling students 


Formal assessments 












Core or supplementary program assessment 


38 


2.95 


28 


2.48 


.006 


District assessment 


4 


1.02 


5 


1.29 


.391 


Standardized assessment 


37 


2.72 


30 


2.58 


.089 


State-specific assessment 


14 


2.01 


13 


1.83 


.625 


Informal assessments 


26 


2.74 


34 


2.72 


.041 


Not able to categorize 


35 


2.97 


50 


2.87 


.000 



Source: Teacher Survey, Question C7. 

^ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three assessments 
per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment was counted only 
once (e.g., if the DIB ELS Nonsense Word Fluency test and the DIB ELS Phonemic Segmentation Fluency test were used to place 
or group students, then the DIB ELS was counted only once for that purpose). 

We were not able to categorize 35-40 percent of teachers’ responses. “Not able to categorize” includes responses that were too 
vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, skill 
test, fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); and 
responses that were not discernable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language and 
Literacy Assessment). 

In order to be grouped as a “district assessment,” the teacher had to clearly label it as a district test. 
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Exhibit C.5.4a 



Types of Specific Assessments (Formai and informai) Reported as Usefui by Mature Reading 
First and Titie i Kindergarten Teachers, by Assessment Purpose, 2004-05 Schooi Year 


Assessment Purpose 

Type of Assessment 


Reading First Teachers 


Title 1 Teachers 




Percent 


Standard 

Error 


Percent 


Standard 

Error 


p-value 


Placing or grouping of students 


Formal assessments 












Clay Observational Survey 


CO 


1.50 


16% 


2.06 


.001 


DIBELS 


43 


2.51 


13 


1.83 


.000 


DRA 


6 


1.27 


13 


1.72 


.001 


STAR Reading 


2 


0.77 


3 


0.81 


.377 


Informal assessments 












Classroom-based assessment ^ 


26 


2.59 


32 


2.54 


.128 


Running records or miscue analysis ® 


8 


1.53 


10 


1.71 


.232 


Determining student mastery of skills 


Formal assessments 












Clay Observational Survey 


6 


1.16 


17 


2.03 


.000 


DIBELS 


33 


2.41 


12 


1.73 


.000 


DRA 


3 


0.88 


11 


1.71 


.000 


STAR Reading 


1 


0.55 


2 


0.66 


.234 


Informal assessments 












Classroom-based assessment 


31 


2.70 


41 


2.67 


.013 


Running records or miscue analysis 


4 


1.19 


7 


1.25 


.068 


Identifying the core deficits of struggling students 


Formal assessments 












Clay Observational Survey 


8 


1.50 


19 


2.20 


.000 


DIBELS 


34 


2.50 


12 


1.71 


.000 


DRA 


4 


1.01 


10 


1.59 


.001 


STAR Reading 


1 


0.61 


2 


0.78 


.239 


Informal assessments 












Classroom-based assessment 


24 


2.58 


38 


2.67 


.000 


Running records or miscue analysis 


6 


1.41 


10 


1.69 


.033 



Source: Teacher Survey, Question C7. 

““ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three 



assessments per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment 
was counted only once (e.g., if the DIB ELS Nonsense Word Fluency test and the DIB ELS Phonemic Segmentation Fluency 
test were used to place or group students, then the DIBELS was counted only once for that purpose). 

’’ We were not able to categorize 35-40 percent of teachers’ responses. “Not able to categorize” includes responses that were 
too vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, 
skill test, fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); 
and responses that were not discemable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language 
and Literacy Assessment). 
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Exhibit C.5.4b 



Types of Specific Assessments (Formai and informai) Reported as Usefui by Mature Reading First 
and Titie i fsf-grade Teachers, by Assessment Purpose, 2004-05 Schooi Year 





Reading First Teachers 


Title I Teachers 


Assessment Purpose 


Standard 


Standard 


Type of Assessment 


Percent Error 


Percent Error p-value 


Placing or grouping of students 



Formal assessments 












Clay Observational Survey 


7% 


1.45 


9% 


1.57 


.336 


DIBELS 


38 


2.53 


10 


1.53 


.000 


DRA 


15 


1.90 


25 


2.22 


.000 


STAR Reading 


6 


1.35 


11 


1.58 


.007 


Informal assessments 












Classroom-based assessment ^ 


15 


2.13 


16 


1.80 


.616 


Running records or miscue analysis ® 


15 


2.08 


29 


2.46 


.000 


Determining student mastery of skills 


Formal assessments 












Clay Observational Survey 


4 


1.02 


9 


1.62 


.006 


DIBELS 


30 


2.37 


7 


1.35 


.000 


DRA 


7 


1.33 


15 


1.88 


.000 


STAR Reading 


3 


1.10 


8 


1.46 


.017 


Informal assessments 












Classroom-based assessment 


19 


2.21 


23 


2.19 


.179 


Running records or miscue analysis 


11 


1.90 


19 


2.25 


.004 



Identifying the core deficits of struggling students 

Formal assessments 



Clay Observational Survey 


4 


1.14 


13 


1.83 


.000 


DIBELS 


39 


2.46 


12 


1.75 


.000 


DRA 


8 


1.49 


16 


1.94 


.001 


STAR Reading 


2 


0.84 


6 


1.22 


.022 


Informal assessments 

Classroom-based assessment 


12 


1.93 


20 


2.14 


.005 


Running records or miscue analysis 


13 


1.96 


26 


2.47 


.000 



Source: Teacher Survey, Question C7. 

““ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three assessments 
per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment was counted only 
once (e.g., if the DIBELS Nonsense Word Fluency test and the DIBELS Phonemic Segmentation Fluency test were used to place 
or group students, then the DIBELS was counted only once for that purpose). 

*’ We were not able to categorize 35-40 percent of teachers’ responses. “Not able to categorize” includes responses that were too 
vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, skill test, 
fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); and responses 
that were not discemable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language and Literacy 
Assessment). 
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Exhibit C.5.4c 



Types of Specific Assessments (Formai and informai) Reported as Usefui by Mature Reading First 
and Titie i 2/7d-grade Teachers, by Assessment Purpose, 2004-05 Schooi Year 





Reading First Teachers 


Title 1 Teachers 




Assessment Purpose 

Type of Assessment 


Percent 


Standard 

Error 


Percent 


Standard 

Error 


p-value 


Placing or grouping of students 


Formal assessments 












Clay Observational Survey 


3% 


1.25 


5% 


1.54 


.310 


DIBELS 


41 


2.54 


9 


1.55 


.000 


DRA 


12 


1.76 


25 


2.29 


.000 


STAR Reading 


7 


1.41 


17 


1.79 


.000 


Informal assessments 












Classroom-based assessment ^ 


15 


2.12 


19 


2.13 


.204 


Running records or miscue analysis ® 


12 


1.82 


21 


2.33 


.004 


Determining student mastery of skills 


Formal assessments 












Clay Observational Survey 


3 


1.22 


4 


1.15 


.474 


DIBELS 


23 


2.21 


6 


1.30 


.000 


DRA 


6 


1.19 


14 


1.97 


.000 


STAR Reading 


3 


0.87 


8 


1.31 


.002 


Informal assessments 












Classroom-based assessment 


23 


2.51 


25 


2.43 


.596 


Running records or miscue analysis 


7 


1.38 


14 


2.05 


.005 


Identifying the core deficits of struggling students 


Formal assessments 












Clay Observational Survey 


5 


1.49 


5 


1.17 


.904 


DIBELS 


30 


2.41 


10 


1.67 


.000 


DRA 


9 


1.49 


18 


2.16 


.001 


STAR Reading 


2 


0.80 


9 


1.50 


.000 


Informal assessments 












Classroom-based assessment 


15 


2.10 


21 


2.39 


.075 


Running records or miscue analysis 


14 


2.05 


22 


2.43 


.006 



Source: Teacher Survey, Question C7. 

“ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three assessments 
per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment was counted only 
once (e.g., if the DIBELS Nonsense Word Fluency test and the DIBELS Phonemic Segmentation Eluency test were used to place 
or group students, then the DIBELS was counted only once for that purpose). 

We were not able to categorize 35-40 percent of teachers’ responses. “Not able to categorize” includes responses that were too 
vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, skill 
test, fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); and 
responses that were not discernable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language and 
Literacy Assessment). 
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Exhibit C.5.4d 



Types of Specific Assessments (Formai and informai) Reported as Usefui by Mature Reading 
First and Titie i 3fd-grade Teachers, by Assessment Purpose, 2004-05 Schooi Year 



Assessment Purpose 

Type of Assessment 


Reading First Teachers 


Title 1 Teachers 




Percent 


Standard 

Error 


Percent 


Standard 

Error 


p-value 


Placing or grouping of students 


Formal assessments 












Clay Observational Survey 


2% 


0.56 


4% 


0.99 


.062 


DIBELS 


35 


2.49 


7 


1.25 


.000 


DRA 


12 


1.75 


17 


1.97 


.045 


STAR Reading 


9 


1.71 


19 


2.00 


.000 


Informal assessments 












Classroom-based assessment ^ 


18 


2.36 


16 


2.10 


.501 


Running records or miscue analysis ® 


6 


1.29 


15 


2.18 


.000 


Determining student mastery of skills 


Formal assessments 












Clay Observational Survey 


3 


1.00 


5 


1.26 


.228 


DIBELS 


19 


2.03 


3 


0.95 


.000 


DRA 


4 


1.03 


8 


1.59 


.050 


STAR Reading 


4 


1.27 


8 


1.41 


.034 


Informal assessments 












Classroom-based assessment 


25 


2.64 


31 


2.59 


.088 


Running records or miscue analysis 


2 


0.67 


8 


1.72 


.003 


Identifying the core deficits of struggling students 


Formal assessments 












Clay Observational Survey 


3 


0.98 


5 


1.27 


.108 


DIBELS 


28 


2.41 


7 


1.34 


.000 


DRA 


9 


1.59 


11 


1.78 


.412 


STAR Reading 


4 


1.36 


9 


1.59 


.012 


Informal assessments 












Classroom-based assessment 


20 


2.53 


23 


2.46 


.364 


Running records or miscue analysis 


8 


1.63 


14 


2.08 


.050 



Source: Teacher Survey, Question C7. 

“ Percentages by grade or assessment purpose will not add up to 100 percent. Teachers were asked to list up to three assessments 
per purpose. If a teacher listed the same assessment more than once for the same purpose, then that assessment was counted only 
once (e.g., if the DIBELS Nonsense Word Fluency test and the DIBELS Phonemic Segmentation Fluency test were used to 
place or group students, then the DIBELS was counted only once for that purpose). 

We were not able to categorize 35-40 percent of teachers’ responses. “Not able to categorize” includes responses that were too 
vague to be coded (e.g., assessment, test, eight-weeks, benchmarks, letter names, decoding, rubric, pretest, quarterly test, skill 
test, fluency); responses that accounted for less than 1 percent of the total frequency (e.g., MGLS, parent reading, KIA); and 
responses that were not discernable (e.g., ELLA stands for Early Learning Literacy in Arkansas or the English Language and 
Literacy Assessment). 
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Exhibit C.5.5 

Regularly Scheduled and Formal Time Set Aside for Grade-Level Teachers to Use 
Assessment Data to Plan Instruction, Range across Grade Levels for Mature Reading First 
and Title I Schools, 2004-05 School Year 

Reading First 

Teachers Titie I Teachers 

Standard Standard 



Frequency of Time Set Aside 


Grade 


Percent 


Error 


Percent 


Error 


p-value 


Once a week or more 


K 


27% 


2.49 


24% 


2.26 


.366 




1 


29 


2.43 


25 


2.21 


.214 




2 


25 


2.25 


25 


2.42 


.964 




3 


27 


2.49 


24 


2.15 


.391 


Once a month 


K 


21 


2.32 


12 


1.67 


.002 




1 


21 


2.28 


14 


1.76 


.024 




2 


24 


2.42 


16 


1.83 


.006 




3 


21 


2.28 


14 


1.77 


.019 


5-8 times 


K 


10 


1.75 


9 


1.61 


.610 




1 


12 


2.04 


9 


1.52 


.162 




2 


11 


1.87 


8 


1.38 


.130 




3 


14 


2.13 


9 


1.67 


.042 


1-4 times 


K 


21 


2.22 


25 


2.21 


.174 




1 


25 


2.42 


28 


2.29 


.390 




2 


23 


2.37 


28 


2.30 


.100 




3 


27 


2.61 


28 


2.33 


.617 


Not at aii 


K 


10 


1.89 


11 


1.61 


.672 




1 


4 


1.07 


12 


1.79 


.000 




2 


6 


1.32 


11 


1.66 


.007 




3 


4 


1.03 


15 


1.89 


.000 


Informaiiy, oniy as needed 


K 


11 


1.80 


18 


2.02 


.003 




1 


9 


1.70 


13 


1.62 


.187 




2 


12 


1.86 


12 


1.63 


.798 




3 


7 


1.40 


10 


1.62 


.158 



Source: Teacher Survey, Question C3. 
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Exhibit C.5.6 

Teachers’ Use of Assessment-Reiated Teaching Strategies, Range across Grade Leveis for 
Mature Reading First and Titie i Schoois, 2004-05 Schooi Year 

Reading First 

Teachers Title I Teachers 



Assessment-Related Teaching 
Strategy 


Grade 


Percent 


Standard 

Error 


Percent 


Standard 

Error 


p-value 


Use test results to organize 


K 


84% 


2.15 


67% 


2.31 


.000 


instructional groups 


1 


85 


2.06 


79 


2.00 


.064 




2 


82 


2.23 


75 


2.21 


.037 




3 


81 


2.34 


70 


2.35 


.001 


Use informal reading inventories 


K 


55 


2.78 


60 


2.48 


.171 




1 


62 


2.70 


68 


2.32 


.138 




2 


54 


2.83 


62 


2.51 


.023 




3 


53 


2.79 


55 


2.60 


.704 


Use tests to determine progress on 


K 


82 


2.14 


73 


2.12 


.003 


skills 


1 


88 


1.82 


80 


1.93 


.005 




2 


83 


2.03 


78 


2.17 


.092 




3 


86 


2.09 


79 


2.16 


.034 


Use tests to determine who can 


K 


54 


2.83 


35 


2.48 


.000 


benefit from the core reading series 


1 


52 


2.82 


49 


2.50 


.531 




2 


56 


2.81 


47 


2.53 


.025 




3 


54 


2.86 


47 


2.61 


.080 


Use diagnostic tests to identify 


K 


72 


2.52 


52 


2.50 


.000 


students who need reading 


1 


77 


2.39 


69 


2.32 


.014 


intervention services 


2 


76 


2.48 


67 


2.36 


.015 




3 


74 


2.59 


69 


2.36 


.122 


Use screening tests to identify 


K 


50 


2.77 


40 


2.53 


.010 


students who need a supplementary 


1 


51 


2.75 


55 


2.54 


.357 


reading program 


2 


52 


2.81 


49 


2.55 


.489 




3 


51 


2.78 


49 


2.61 


.598 


Conduct miscue analysis, analyzing 


K 


36 


2.71 


42 


2.58 


.099 


errors students make while reading 


1 


56 


2.76 


63 


2.41 


.057 


aloud 


2 


55 


2.81 


56 


2.57 


.721 




3 


52 


2.82 


48 


2.62 


.259 



Source: Teacher Survey, Question C6. 
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Exhibit C.7.1 

Structure of Professionai Deveiopment Activities Attended by Teachers in Mature Reading First 
and Titie i Schoois, 2004-05 Schooi Year 



Reading First Teachers Title I Teachers 





Mean # 
Workshops 


Mean # 
Hours 


Mean # 
Workshops 


Mean # 
Hours 


Workshop 

p-vaiue 


Hours 

p-vaiue 


D1a. Short workshops 


Grade: K 


3.88 


10.51 


2.60 


6.45 


0.0025 


0.0020 


1 


4.34 


10.74 


2.51 


7.01 


0.0000 


0.0025 


2 


4.14 


10.49 


2.12 


5.71 


0.0000 


0.0000 


3 


3.99 


9.84 


2.68 


6.82 


0.0000 


0.0030 


D1b. Long workshops 


Grade: K 


2.31 


24.78 


1.34 


11.14 


0.0000 


0.0000 


1 


1.80 


21.99 


2.65 


13.42 


0.0041 


0.0000 


2 


2.00 


23.27 


1.16 


9.86 


0.0000 


0.0000 


3 


2.19 


22.31 


1.14 


9.59 


0.0000 


0.0000 



D1c. College course In 
reading 



Grade: K 


0.42 


10.10 


0.35 


8.22 


0.4948 


0.6137 


1 


0.41 


7.11 


0.44 


7.40 


0.7572 


0.8715 


2 


0.69 


8.03 


0.21 


4.82 


0.0130 


0.1592 


3 


0.48 


6.99 


0.61 


7.31 


0.5908 


0.8916 


Did. Conference about 














reading 














Grade: K 


0.81 


5.82 


0.73 


3.14 


0.6302 


0.0021 


1 


0.67 


5.09 


0.69 


3.89 


0.8659 


0.1376 


2 


0.71 


6.17 


0.55 


2.79 


0.1869 


0.0010 


3 


0.99 


6.81 


0.50 


3.79 


0.0049 


0.0139 



Source: Teacher survey, Question Dl. 
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Exhibit C.7.3 



Specific Features of Professionai Deveiopment Activities Differences between Teachers in 
Mature Reading First and Titie i Schoois. 2004-05 Schooi Year 





Reading First Teachers 


Title 1 Teachers 




Specific Features 


Percent 


Standard 

Error 


Percent 


Standard 

Error 


p-vaiue 


Required teachers te attend 


74% 


1.07 


57% 


1.33 


0.000 


Were attended by the 
principal 


52 


1.35 


46 


1.25 


0.000 


Previded teachers with 
options 


33 


1.27 


32 


1.04 


0.542 


Provided a stipend 


40 


1.41 


20 


1.11 


0.000 


Provided follow-up activities 


48 


1.30 


34 


1.09 


0.000 


Included release time 


43 


1.35 


33 


1.16 


0.000 


Offered graduate college 
credits 


25 


1.20 


14 


0.80 


0.000 


Were held in convenient 
location 


73 


1.11 


62 


1.20 


0.000 


Used team-based approach 


67 


1.06 


53 


1.20 


0.000 


Were given by well- 
established trainers 


75 


1.06 


65 


1.17 


0.000 



Source: Teacher Survey, Question D3. 
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Exhibit C.7.4 



Teacher Participation in Professionai Deveiopment Activities Reiated to the Five 
Dimensions of Reading: Teachers in Mature Reading First and Titie i Schoois, 2004-05 
Schooi Year 



PD Participation 



Dimension 


Grade 


Reading First Teachers 


Title 1 Teachers 


p- value 


Percent 


Standard 

Error 


Percent 


Standard 

Error 


Phonemic 


K 


92% 


0.016 


74% 


0.022 


0.000 


Awareness 


1 


89 


0.018 


70 


0.023 


0.000 




2 


82 


0.021 


53 


0.026 


0.000 




3 


75 


0.025 


50 


0.026 


0.000 


Deceding 


K 


91 


0.015 


72 


0.023 


0.000 




1 


86 


0.019 


68 


0.023 


0.000 




2 


85 


0.019 


57 


0.026 


0.000 




3 


81 


0.022 


54 


0.026 


0.000 


Vocabulary 


K 


67 


0.027 


49 


0.026 


0.000 




1 


72 


0.025 


50 


0.025 


0.000 




2 


77 


0.023 


52 


0.026 


0.000 




3 


78 


0.023 


56 


0.026 


0.000 


Fluency 


1 


87 


0.020 


73 


0.022 


0.000 




2 


88 


0.019 


67 


0.024 


0.000 




3 


84 


0.022 


66 


0.024 


0.000 


Comprehension 


K 


87 


0.019 


76 


0.021 


0.000 




1 


85 


0.019 


75 


0.021 


0.001 




2 


90 


0.016 


73 


0.023 


0.000 




3 


88 


0.018 


74 


0.022 


0.000 


Perceived PD Needs 


Phonemic 


K 


51 


0.033 


62 


0.030 


0.020 


Awareness 


1 


25 


0.027 


31 


0.026 


0.093 




2 


31 


0.032 


38 


0.030 


0.105 




3 


24 


0.031 


35 


0.030 


0.009 


Deceding 


K 


71 


0.030 


75 


0.026 


0.276 




1 


45 


0.032 


45 


0.029 


0.867 




2 


38 


0.033 


41 


0.030 


0.495 




3 


34 


0.035 


38 


0.030 


0.375 


Vocabulary 


K 


50 


0.033 


41 


0.030 


0.029 




1 


52 


0.032 


52 


0.029 


0.937 




2 


49 


0.034 


41 


0.030 


0.095 




3 


46 


0.036 


42 


0.032 


0.308 


Fluency 


1 


61 


0.031 


56 


0.030 


0.266 




2 


46 


0.033 


54 


0.030 


0.073 




3 


49 


0.034 


49 


0.031 


0.956 


Comprehension 


K 


71 


0.031 


71 


0.028 


0.845 




1 


75 


0.028 


81 


0.022 


0.087 




2 


81 


0.026 


77 


0.027 


0.227 




3 


82 


0.025 


81 


0.025 


0.786 



Source: Teacher Survey, Question D4. 
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Exhibit C.7.5 



Preparedness to Teach Five Dimensions of Reading: Teachers in Reading First and 
Titie i Schoois, 2004-05 Schooi Year 



Dimension 


Grade 


Reading First Teachers 


Title 1 Teachers 


p-vaiue 


Mean 


Standard 

Error 


Mean 


Standard 

Error 


Phonemic 


K 


4.17 


0.052 


3.66 


0.067 


0.000 


Awareness 


1 


4.09 


0.051 


3.66 


0.066 


0.000 




2 


3.73 


0.060 


3.22 


0.070 


0.000 




3 


3.52 


0.067 


2.95 


0.070 


0.000 


Decoding 


K 


3.99 


0.055 


3.47 


0.067 


0.000 




1 


4.06 


0.054 


3.61 


0.064 


0.000 




2 


3.78 


0.059 


3.30 


0.069 


0.000 




3 


3.60 


0.066 


2.99 


0.075 


0.000 


Vocabulary 


K 


3.81 


0.057 


3.33 


0.067 


0.000 




1 


3.77 


0.059 


3.46 


0.064 


0.000 




2 


3.74 


0.057 


3.43 


0.067 


0.001 




3 


3.84 


0.056 


3.37 


0.066 


0.000 


Comprehension 


K 


3.81 


0.058 


3.46 


0.061 


0.000 




1 


3.79 


0.058 


3.55 


0.061 


0.000 




2 


3.77 


0.058 


3.57 


0.064 


0.023 




3 


3.82 


0.056 


3.55 


0.067 


0.002 


Fluency Building 


K 


3.65 


0.066 


3.25 


0.067 


0.000 




1 


3.73 


0.064 


3.45 


0.064 


0.002 




2 


3.87 


0.058 


3.38 


0.068 


0.000 




3 


3.88 


0.059 


3.38 


0.068 


0.000 



Source: Teacher Survey, Question D6. 



Exhibit C.7.7a 

Teacher Professional Development on Materials and Teaching Strategies in Reading First 
Schools, 2004-05 School Year 




Grade 


Percent 


Standard 

Error 


Received Professionai Deveiopment 








Teaching Using the core reading program 


overall 


84% 


0.008 


Strategies 


K 


84 


0.014 




1 


84 


0.013 




2 


85 


0.016 




3 


82 


0.014 


Using children's literature to teach 


overall 


56 


0.010 


reading 


K 


64 


0.019 




1 


51 


0.019 




2 


58 


0.019 




3 


52 


0.020 


Using reading research to guide 


overall 


62 


0.011 


content of instruction 


K 


64 


0.019 




1 


59 


0.020 




2 


64 


0.018 




3 


62 


0.019 


How the core reading program 


overall 


64 


0.010 


incorporates research principles 


K 


67 


0.018 




1 


63 


0.019 




2 


66 


0.018 




3 


61 


0.020 


Using the supplemental reading 


overall 


51 


0.010 


programs 


K 


47 


0.019 




1 


49 


0.019 




2 


57 


0.019 




3 


53 


0.020 


Integrating reading and writing 


overall 


62 


0.011 
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Exhibit C.7.7a 








Teacher Professionai Deveiopment on Materiais and Teaching Strategies in Reading First 


Schoois, 2004-05 Schooi Year 














Standard 




Grade 


Percent 


Error 


instruction 


K 


63 


0.019 




1 


60 


0.019 




2 


63 


0.019 




3 


61 


0.019 


Strategies for teaching reading to 


overall 


41 


0.011 


ELLs 


K 


42 


0.019 




1 


39 


0.018 




2 


44 


0.019 




3 


41 


0.019 


Need Additional Professional Development in This 








Topic 








Teaching Using the core reading program 


K 


7 


0.011 


Strategies 


1 


8 


0.012 




2 


6 


0.010 




3 


8 


0.014 


Using chiidren's literature to teach 


K 


16 


0.017 


reading 


1 


22 


0.019 




2 


23 


0.021 




3 


22 


0.021 


Using reading research to guide 


K 


12 


0.017 


content of instruction 


1 


12 


0.015 




2 


9 


0.013 




3 


11 


0.016 


How the core reading program 


K 


6 


0.013 


incorporates research principles 


1 


6 


0.011 




2 


7 


0.012 




3 


7 


0.015 


Using the supplemental reading 


K 


21 


0.020 


programs 


1 


17 


0.018 




2 


17 


0.017 




3 


21 


0.020 


Integrating reading and writing 


K 


28 


0.021 


instruction 


1 


28 


0.021 




2 


34 


0.023 




3 


31 


0.023 


Strategies for teaching reading to 


K 


24 


0.019 


ELLs 


1 


25 


0.019 




2 


28 


0.022 




3 


29 


0.023 



Source: Teacher Survey, Question D5. 
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Exhibit C.7.7b 








Teacher Professionai Deveiopment on Grouping and Assessment in Reading First 


Schoois, 2004-05 Schooi Year 














Standard 




Grade 


Percent 


Error 


Received Professionai Deveiopment 


Grouping Learning styies 


overall 


48% 


0.010 




K 


51 


0.019 




1 


45 


0.019 




2 


50 


0.020 




3 


45 


0.020 


Organizing smali group instruction 


overall 


69 


0.010 




K 


66 


0.019 




1 


69 


0.019 




2 


72 


0.018 




3 


67 


0.020 


Assessment Diagnosing reading problems 


overall 


49 


0.010 




K 


45 


0.019 




1 


46 


0.019 




2 


54 


0.020 




3 


49 


0.020 


Administering assessments 


overall 


76 


0.009 




K 


74 


0.018 




1 


74 


0.017 




2 


79 


0.016 




3 


76 


0.017 


Interpreting and using assessment 


overall 


78 


0.008 


data to guide instruction 


K 


75 


0.017 




1 


77 


0.016 




2 


81 


0.015 




3 


77 


0.017 


Need Additional Professional Development in This 








Topic 








Grouping Learning styles 


K 


25 


0.020 




1 


20 


0.018 




2 


23 


0.020 




3 


20 


0.020 


Organizing small group instruction 


K 


25 


0.020 




1 


21 


0.019 




2 


23 


0.021 




3 


24 


0.022 


Assessment Diagnosing reading problems 


K 


40 


0.023 




1 


41 


0.022 




2 


36 


0.023 




3 


33 


0.024 


Administering assessments 


K 


5 


0.009 




1 


3 


0.007 




2 


3 


0.008 




3 


5 


0.011 


Interpreting and using assessment 


K 


11 


0.013 


data to guide instruction 


1 


11 


0.013 




2 


9 


0.013 




3 


12 


0.016 


Source: Teacher Survey, Question D5. 
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Exhibit C.7.7c 

Teacher Professionai Deveiopment on Struggiing Readers in Reading First Schoois, 2004- 
05 Schooi Year 

Standard 





Grade 


Percent 


Error 


Received Professionai Deveiopment 


Struggling Helping struggling readers with 


overall 


64% 


0.010 


Readers decoding 


K 


59 


0.019 




1 


66 


0.019 




2 


68 


0.018 




3 


61 


0.020 


Helping struggling readers with 


overall 


62 


0.010 


vocabulary 


K 


57 


0.019 




1 


61 


0.019 




2 


68 


0.018 




3 


63 


0.019 


Helping struggling readers with 


overall 


60 


0.010 


comprehension 


K 


56 


0.019 




1 


58 


0.019 




2 


64 


0.019 




3 


59 


0.020 


Motivating readers 


overall 


48 


0.010 




K 


48 


0.019 




1 


47 


0.019 




2 


50 


0.020 




3 


47 


0.020 


Teaching reading to students with 


overall 


28 


0.009 


learning disabilities 


K 


26 


0.017 




1 


25 


0.016 




2 


33 


0.019 




3 


29 


0.018 


Need Additional Professional Development 








Struggling Helping struggling readers with 


K 


33 


0.022 


Readers decoding 


1 


27 


0.020 




2 


21 


0.019 




3 


19 


0.020 


Helping struggling readers with 


K 


15 


0.016 


vocabulary 


1 


21 


0.019 




2 


18 


0.018 




3 


18 


0.019 


Helping struggling readers with 


K 


24 


0.020 


comprehension 


1 


33 


0.022 




2 


33 


0.022 




3 


33 


0.024 


Motivating readers 


K 


26 


0.020 




1 


28 


0.021 




2 


33 


0.023 




3 


32 


0.023 


Teaching reading to students with 


K 


42 


0.023 


learning disabilities 


1 


43 


0.023 




2 


43 


0.024 




3 


38 


0.024 



Source: Teacher Survey, Question D5. 
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Exhibit C.7.7d 

Teacher Professionai Deveiopment on Organization and Pianning in Reading First 
Schoois, 2004-05 Schooi Year 




Grade 


Percent 


Standard 

Error 


Received Professionai Deveiopment in This Topic 


Organization/ Using state/district content 


everaii 


68% 


0.010 


Pianning standards for curricuium pianning 


K 


68 


0.018 


and teaching 


1 


66 


0.019 




2 


71 


0.017 




3 


69 


0.018 


Aiigning reading curricuium and 


everaii 


66 


0.010 


instruction with state/district 


K 


65 


0.019 


assessments 


1 


63 


0.019 




2 


69 


0.018 




3 


66 


0.019 


Working with parents 


everaii 


35 


0.010 




K 


34 


0.018 




1 


30 


0.018 




2 


38 


0.019 




3 


36 


0.020 


Ciassreem management 


everaii 


56 


0.011 




K 


54 


0.019 




1 


54 


0.020 




2 


60 


0.019 




3 


57 


0.020 


Need Additional Professional Development in This 
Topic 


Organization/ Using state/district content 


K 


8 


0.014 


Pianning standards for curricuium pianning 


1 


6 


0.011 


and teaching 


2 


7 


0.012 




3 


7 


0.012 


Aiigning reading curricuium and 


K 


8 


0.012 


instruction with state/district 


1 


8 


0.013 


assessments 


2 


9 


0.013 




3 


9 


0.012 


Working with parents 


K 


24 


0.021 




1 


22 


0.019 




2 


20 


0.019 




3 


22 


0.020 


Ciassreem management 


K 


17 


0.018 




1 


17 


0.016 




2 


14 


0.017 




3 


17 


0.020 


Source: Teacher Survey, Question D5. 
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Exhibit C.7.8 



Professionai Deveiopment in Teaching Strategies in Mature Reading First and Titie i 
Schoois, 2004-05 Schooi Year 







Reading First Teachers 


Title 1 Teachers 




Topics 


Grade 


Percent 


Standard 

Error 


Percent 


Standard 

Error 


p- value 


Teaching 


K 


92% 


0.015 


72% 


0.022 


0.000 


Strategies 


1 


90 


0.016 


71 


0.022 


0.000 




2 


92 


0.016 


71 


0.023 


0.000 




3 


89 


0.018 


74 


0.022 


0.000 


Grouping 


K 


64 


0.027 


49 


0.025 


0.000 




1 


65 


0.027 


50 


0.025 


0.000 




2 


66 


0.026 


52 


0.026 


0.000 




3 


65 


0.028 


45 


0.026 


0.000 


Assessment 


K 


82 


0.023 


64 


0.024 


0.000 




1 


83 


0.020 


62 


0.024 


0.000 




2 


89 


0.018 


68 


0.023 


0.000 




3 


84 


0.020 


64 


0.025 


0.000 


Struggiing 


K 


69 


0.026 


54 


0.025 


0.000 


Readers 


1 


73 


0.025 


60 


0.024 


0.000 




2 


79 


0.022 


53 


0.026 


0.000 




3 


76 


0.024 


54 


0.026 


0.000 


Organization/ 


K 


62 


0.027 


56 


0.025 


0.115 


Pianning 


1 


60 


0.027 


53 


0.025 


0.052 




2 


67 


0.026 


53 


0.026 


0.000 




3 


67 


0.027 


57 


0.026 


0.009 



Source: Teacher Survey, Question D5. 
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Exhibit C.7.9 

RF Principais’ Participation in Professionai Deveiopment in Teaching Strategies in 



Reading First Schoois, 2004-05 Schooi Year 






Professional Development Topic 


Percent 


Standard Error 


Content of Reading 


Phonemic Awareness 


84% 


0.013 


Instruction 


Decoding 


78 


0.014 




Vocabulary 


84 


0.013 




Fluency 


87 


0.012 




Comprehension 


86 


0.012 


Teaching Strategies 


How to use the cere reading program 


75 


0.014 




How to use children’s literature to 
teach reading 


43 


0.016 




Hew to use reading research to guide 
content of instruction 


63 


0.016 




How the core reading program 
incorporates research principles 


62 


0.016 




How to use the supplemental reading 
pregram 


56 


0.016 




Hew to integrate reading and writing 
instruction 


58 


0.017 




Strategies for teaching reading to 
ELLs 


43 


0.016 


Evaluation 


How to evaluate a core reading 
pregram 


42 


0.016 




Hew to evaluate reading instruction 


66 


0.016 




How to coach teachers in reading 
instruction 


52 


0.017 




How to manage reading personnel 


40 


0.016 


Assessment 


How to diagnose reading problems 


50 


0.017 




Hew to administer assessments 


67 


0.016 




How to interpret and use assessment 
data to guide instructien 


84 


0.012 


Struggling Readers 


Hew to help struggling readers with 
deceding 


58 


0.017 




How to help struggling readers with 
vocabulary 


63 


0.016 




How to help struggling readers with 
comprehension 


60 


0.017 




How to motivate readers 


48 


0.017 




Strategies for teaching reading to 
students with diagnosed learning 
disabilities 


35 


0.016 


Organization, 


How to select reading materials 


36 


0.016 


Management and 
Support 


Hew to use content standards fer 
curriculum planning and teaching 


57 


0.017 




How select reading assessments 


32 


0.015 




Alignment ef reading curriculum and 
instruction state/district assessments 


61 


0.016 




Hew to work with parents 


36 


0.016 




Classroom management 


47 


0.017 




Other 


7 


0.014 



Source: Principal Survey, Question F2. 
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Exhibit D.1 



Construct Variables for Analysis of Survey Data 



Construct 


Survey 


Survey 


Survey Description 


item Scaie 


Scoring Metric 




Question 


Item 









Support for Struggiing Readers 



Support for 

Struggling 

Readers® 


Grade 1 
Teacher 
Survey, C9 


b. 


Extra practice with phonemic awareness 


Dichotomous 
1 = “Received” 
0 = “Did not 


Score 1 for each 
Survey Item checked 
as “Received.” 




c. 


Extra practice with decoding/phonics 


receive” 


Sum Survey Item 






d. 


Extra practice with fluency 




scores 












(Range: 0-3) 



High Fideiity (SBRR) Reading instruction— KiNDERGARTEN (K) 



KINDERGARTEN 
High Fidelity 
Vocabulary 
Instruction 


K Teacher 
Survey, C4 


bb. 


I discuss meaning of new and unusual words 


Three-level ordinal 


Score 1 if Survey 
Item checked 
“Central to my 
instruction.” 


dd. 


Students give definitions for words 


1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 








ee. 


Students tell opposites of words 


Compute percent of 
items in construct 
rated as “central.” 


3 = “Not part of my 
reading instruction” 












(Range: 0 to 100%) 



a 



We created two versions of several constructs: a ‘strict’ version and a more ‘relaxed’ version. Our hope is to use the ‘strict’ version of the construct, as we 
believe these represent the highest quality activities. However, it could be the case that we find little variation across these strict constructs, especially in our 
sample of new RF schools. Therefore, the more relaxed versions may be more appropriate for analysis as they allow for more variation across respondents. 
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Construct 


Survey 

Question 


Survey 

Item 


Survey Description 


Item Scale 


Scoring Metric 


High Fidelity (SBRR) Reading Instruction— 


-KINDERGARTEN (K) (continued) 






KINDERGARTEN 
High Fidelity 


K Teacher 
Survey, C4 


b. 


1 conduct story discussions with small groups of 
students 


Three-level ordinal 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 


Comprehension 




c. 


1 read stories to small groups of students 




Instruction 




g- 


Students orally answer questions about stories they 
have heard 


1 = “Central to my 
reading instruction” 






h. 


Students read texts that are easy to decode 


2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction” 


Compute percent of 






k. 


Students create story maps based on stories read aloud 


items in construct 






y- 


Students retell stories in sequence and identify 
characters and main events 


rated as “central.” 






gg- 


Students make predictions while reading stories 




(Range: 0 to 100%) 


KINDERGARTEN 


K Teacher 


a. 


1 read stories aloud to the whole class 


Three-level ordinal 




Non-SBRR 


Survey, C4 


e. 


1 develop language experience stories with my class 




Score 1 for each Survey 


Instruction 




f. 


Students read aloud their own written dictation 


1 = “Central to my 


Item checked “Central to 






1. 


1 teach phonemic awareness skills while reading stories 


reading instruction” 


my instruction.” 






q- 


Students practice naming letters 


2 = “Smali part of 








V. 


1 teach sight words 


reading instruction” 


Sum Survey Item scores 






cc. 


Students tell opposites of words 


3 = “Not part of my 








hh. 


Students use dictionaries to find word meanings 


reading instruction” 


(Range: 0 to 9) 


KINDERGARTEN 


K Teacher 


m. 


Students isolate first sounds in words that 1 say 


Three-level ordinal 


Score 1 for each Survey 


High Fidelity 


Survey, C4 


n. 


Students isolate final sounds in words that 1 say 


1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 


Item checked “Central to 


Phonemic 

Awareness/Phonics 

Instruction 




0. 


Students isolate middle sounds in words that 1 say 


my instruction.” 




p- 


Students blend sounds with rhyming words 


Sum Survey Item scores 




r. 


Students match sounds with letters 






s. 


Students blend sounds to form words 


3 = “Not part of my 


Compute percent of 






t. 


1 practice identifying sounds and syllables in spoken words by 
clapping and counting 


reading instruction” 


items in construct rated 
as “central.” 












(Range: 0 to 1 00%) 
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Construct 


Survey 

Question 


Survey 

Item 


Survey Description 


Item Scale 


Scoring Metric 


High Fidelity (SBRR) Reading Instruction— 


-KINDERGARTEN (K) (continued) 






KINDERGARTEN 


K Teacher 


a. 


Provide time in reading block for skill practice on own 


Three-level ordinal 


Score 1 for each 


High Fidelity 
Teaching 


Survey, C6 


c. 


Provide extra reading instructional time for struggling 
readers 


1 = “Central to my 


Survey Item checked 
“Central to my 


Strategies 




i. 


Work with small groups of students 


reading instruction” 


instruction.” 






j- 


Group students based on skill levels 


2 = “Small part of 


Compute percent of 






m. 


Use core reading series 


reading instruction” 






n. 


Use supplementary reading materials 


3 = “Not part of my 


items in construct 






r. 


Use separate intervention materials for some students 


reading instruction” 


rated as “central.” 






u. 


Use test results to organize instructional groups 




(Range: 0 to 100%) 






w. 


Use tests to determine progress on skills 








y- 


Use diagnostic tests to identify students who need 
reading intervention services 







High Fidelity (SBRR) Reading Instruction — GRADE 1 



GRADE 1 High 
Fidelity Fluency 
Instruction 


Grade 1 
Teacher 
Survey, C4 


d. 


Students reread familiar stories 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction” 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 

Sum Survey Item 






h. 


Students read aloud with expression and proper 
phrasing 


scores 

Compute percent of 
items in construct 
rated as “central.” 












(Range: 0 to 100%) 


GRADE 1 High 
Fidelity Vocabulary 
Instruction (No 
construct — only 
individual item) 


Grade 1 
Teacher 
Survey, C4 


z. 


1 discuss new and unusual words before reading 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction 


Score 1 00% if 
Survey Item 
checked “Central to 
my instruction.” 

Score 0% if item 
scored 2 or 3. 
(Range: 0 to 100%) 
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Construct 


Survey 

Question 


Survey 

Item 


Survey Description 


Item Scale 


Scoring Metric 


High Fidelity (SBRR) Reading Instruction — GRADE 1 (continued) 


GRADE 1 Non- 
SBRR Instruction 


Grade 1 
Teacher 
Survey, C4 


c. 


Students read silently 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction” 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 

Sum Survey Item 
scores 

Compute percent of 
items in construct 
rated as “central.” 

(Range: 0 to 100%) 


m. 


Students practice naming letters 


P- 


Students use knowledge of root words, prefixes, and 
suffices to decode new words 


r. 


Students use pictures to identify unknown words 


s. 


I teach decoding skills while reading stories 


GRADE 1 High 
Fidelity 

Comprehension 

Instruction 


Grade 1 
Teacher 
Survey, C4 


i. 


Students reread to find facts to answer questions 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction” 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 

Sum Survey Item 
scores 

Compute percent of 
items in construct 
rated as “central.” 

(Range: 0 to 100%) 


w. 


Students retell stories in sequence and identify 
characters and main events 


cc. 


Students make predictions while reading stories 


ft. 


Students develop questions about text material 
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Construct 


Survey 

Question 


Survey 

Item 


Survey Description 


Item Scale 


Scoring Metric 


High Fidelity (SBRR) Reading Instruction— 


-GRADE 1 (continued) 






GRADE 1 High 
Fidelity Phonemic 
Awareness/Phonics 


Grade 1 
Teacher 
Survey, C4 


I. 


Students isolate sounds in words that teachers say 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction” 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 

Sum Survey Item 
scores 

Compute percent of 
items in construct 
rated as “central.” 


Instruction 


n. 


Students blend phonemes to form words 






u. 


Teaches decoding/phonics skills with word families 












(Range: 0 to 100%) 


GRADE 1 High 


Grade 1 


a. 


Provide time in reading block for skill practice on own 


Three-level ordinal 


Score 1 for each 


Fidelity Teaching 
Strategies 


Teacher 
Survey, C6 


c. 


Provide extra reading instructional time for struggling 
readers 


1 = “Central to my 


Survey Item checked 
“Central to my 






i. 


Work with small groups of students 


reading instruction” 


instruction.” 






j- 


Group students based on skill levels 


2 = “Small part of 


Sum Survey Item 






m. 


Use core reading series 


reading instruction” 






n. 


Use supplementary reading materials 


3 = “Not part of my 


scores 






r. 


Use separate intervention materials for some students 


reading instruction” 


Compute percent of 
items in construct 
rated as “central.” 

(Range: 0 to 100%) 






u. 


Use test results to organize instructional groups 








w. 


Use tests to determine progress on skills 








y- 


Use diagnostic tests to identify students who need 
reading intervention services 
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Construct 


Survey 


Survey 


Survey Description 


Item Scale 


Scoring Metric 




Question 


Item 









High Fidelity (SBRR) Reading Instruction — GRADE 2/3 



GRADE 2/3 High 
Fidelity Fluency 
Instruction 


Grade 2/3 
Teacher 
Survey, C4 


d. 


Students reread familiar stories 


Three-level ordinal 

1 = “Central to my 
reading instruction” 


Score 1 for each 
Survey Item 
checked “Central to 
my instruction.” 






g- 


1 listen to students read aloud without correcting errors 


2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction” 


Sum Survey Item 
scores 

Compute percent of 
items in construct 
rated as “central.” 












(Range: 0 to 100%) 


GRADE 2/3 High 
Fidelity Vocabulary 
Instruction 


Grade 2/3 
Teacher 
Survey, C4 


t. 


Students work with prefixes and suffixes to change the 
meaning of words 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 

3 = “Not part of my 


Score 1 if Survey 
Item checked 
“Central to my 
instruction.” 

Sum Survey Item 
scores 






X. 


Students learn vocabulary through study of antonyms, 
synonyms, and homonyms 






bb. 


1 discuss new and unusual words before reading 


reading instruction” 


Compute percent of 
items in construct 
rated as “central.” 












(Range: 0 to 100%) 
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Construct 


Survey 

Question 


Survey 

Item 


Survey Description 


Item Scale 


Scoring Metric 


High Fidelity (SBRR) Reading Instruction— 


-GRADE 2/3 (continued) 






GRADE 2/3 Non- 
SBRR Instruction 


Grade 2/3 
Teacher 
Survey, C4 


a. 


Students read texts that are easy to decode 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 




c. 


Students read aloud unfamiliar texts 






f. 


Students read silently 


reading instruction” 
3 = “Not part of my 
reading instruction” 


Sum Survey Item 
scores 






0. 


1 teach decoding skills while reading stories 


(Compute percent of 
items in construct 






P- 


Students memorize sight words 




rated as “central.” 






V. 


Students use context clues to identify unknown words 




(Range: 0 to 100%) 






z. 


Students write vocabulary words in sentences 








aa. 


Students use dictionaries to find word meanings 






GRADE 2/3 High 
Fidelity 

Comprehension 

Instruction 


Grade 2/3 
Teacher 
Survey, C4 


b. 


Students reread to locate information 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 


i. 


Students confirm or revise predictions after reading 






j- 


Students generate their own questions about text 
material 


reading instruction” 
3 = “Not part of my 
reading instruction” 


Sum Survey Item 
scores 






k. 


Students identify their comprehension break-downs and 
use fix-up strategies with a partner 


Compute percent of 
items in construct 






1 . 


Students orally summarize main events in stories and 
informational texts 




rated as “central.” 
(Range: 0 to 100%) 






bb. 


1 discuss new and unusual words before reading 








dd. 


Students identify story structure and elements 
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Construct 


Survey 


Survey 


Survey Description 


Item Scale 


Scoring Metric 




Question 


Item 









High Fidelity (SBRR) Reading Instruction — GRADE 2/3 (continued) 



GRADE 2/3 High 
Fidelity Phonemic 
Awareness/Phonics 
Instruction 


Grade 2/3 
Teacher 
Survey, C4 


n. 


Students decode multi-syllabic words in isolation 


Three-level ordinal 

1 = “Central to my 
reading instruction” 

2 = “Small part of 
reading instruction” 

3 = “Not part of my 
reading instruction” 


Score 1 for each 
Survey Item checked 
“Central to my 
instruction.” 

Score -1 for each 
distractor checked 
“Central to my 
instruction.” 

Sum Survey Item 
scores 

Compute percent of 
items in construct 
rated as “central.” 

(Range: 0 to 100%) 


q- 


Students read irregularly spelled words and non-words 


s. 


Students use knowledge of root woods, prefixes, and 
suffixes to decode new words 


GRADE 2/3 High 


Grade 2/3 


a. 


Provide time in reading block for skill practice on own 


Three-level ordinal 


Score 1 for each 


Fidelity Teaching 


Teacher 


c. 


Provide extra reading instructional time for struggling 




Survey Item checked 


Strategies 


Survey, C6 




readers 


1 = “Central to my 


“Central to my 






i. 


Work with small groups of students 


reading instruction” 


instruction.” 






j- 


Group students based on skill levels 


2 = “Small part of 








m. 


Use core reading series 


reading instruction” 


Sum Survey Item 






n. 


Use supplementary reading materials 


3 = “Not part of my 


scores 






r. 


Use separate intervention materials for some students 


reading instruction” 


Compute percent of 






u. 


Use test results to organize instructional groups 








w. 


Use tests to determine progress on skills 




items in construct 






y- 


Use diagnostic tests to identify students who need 




rated as “central.” 








reading intervention services 
















(Range: 0 to 100%) 
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Construct 


Survey 


Survey 


Survey Description 


Item Scale 


Scoring Metric 




Question 


Item 









Professional Development (PD) for Teachers (Applies to all grade levels) 



Diversity of 
Professionai 
Development 
Support 


Teacher 
Survey, D2 


a. 


Coaching or mentoring by reading coach in programs, 
materials, or strategies 


Three level ordinal 

1 = “Received this 
type of assistance” 

2 = “Available, but 
did not receive” 

3 = “Not available 
at my school” 


Score 1 for each 
Survey Item checked 
“Received this type of 
assistance.” 

Sum Survey Item 
scores 

(Range: 0 to 5) 


d. 


Demonstrations in my classroom 


h. 


Interpretation of assessment data 


i. 


Grade-level meetings devoted to reading 


j- 


Using assessment data to determine topics that require 
additional instruction or practice 


Received PD in 

Phonemic 

Awareness 


Teacher 
Survey, D4 


a. 


Building phonological awareness 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items a - c 
are checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


b. 


Identifying, adding, deleting sounds in spoken words 


c. 


Blending phonemes to form words 


Received PD in 
Decoding 


Teacher 
Survey, D4 


d. 


Teaching letter-sound correspondence 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items d - g 
are checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


e. 


Teaching letter patterns 


f. 


Using syllable patterns to read words 


g- 


Teaching component parts 


Received PD in 
Vocabulary 


Teacher 
Survey, D4 


h. 


Teaching use of dictionary, thesaurus 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items h - j 
are checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


i. 


Direct teaching of vocabulary words and meaning 


j- 


Antonyms and synonyms 
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Construct 


Survey 


Survey 


Survey Description 


Item Scale 


Scoring Metric 




Question 


Item 









Professional Development (PD) for Teachers (continued) 



Received PD in 
Fluency 


Teacher 
Survey, D4 


k. 


Teaching sight words 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items k - m 
are checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


I. 


Guided oral reading 


m. 


Encouraging expression while reading 


Received PD in 
Comprehension 


Teacher 
Survey, D4 


n. 


Setting motivation/asking prediction/preview questions 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items n - s 
are checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


0. 


Constructing information about character, setting, and 
main events 


P- 


Summarizing main ideas in narrative and informational 
text 


q- 


Self-monitoring strategies 


r. 


Asking questions at different levels (literal, inferential) 


s. 


Strategies for organizing text structure, e.g. story maps 


Diversity of PD 


Teacher 


a - c. 


Received PD in phonemic awareness construct (see 


Each construct is 


Sum of the above 


Topics Received 


Survey, D4 




above) 


scored as 0 or 1 , 


constructs (which are 






d-g. 


Received PD in decoding/phonics construct (see above) 


“Received PD” or 


scored 0 or 1). 






h-j. 


Received PD in vocabulary construct (see above) 


“Did not Receive 








k- m. 


Received PD in fluency construct (see above) 


PD.” 


(Range: 0 to 5) 






n -s. 


Received PD in comprehension construct (see above) 









D-12 Appendix D 



Construct 


Survey 


Survey 


Survey Description 


Item Scale 


Scoring Metric 




Question 


Item 









Professional Development (PD) for Building Administrators 



Received PD in 
Reading Instruction 
Content 


Principai 
Survey, F2 


a. 


Phonemic Awareness 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 for each 
content area (a-e) 
checked “Topic 
addressed in PD.” 

Sum Survey Item 
scores 

(Range: 0 to 5) 


b. 


Decoding/Phonics 


c. 


Vocabuiary 


d. 


Fluency 


e. 


Comprehension 


Received PD in 

Teaching 

Strategies 


Principai 
Survey, F2 


f. 


How to use the core reading program 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items are 
checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


h. 


How to use reading research to guide content of 
instruction 


i. 


How the core reading program incorporates research 
principals 


j- 


How to use the supplemental reading program 


I. 


Strategies for teaching reading to ELLs 


Received PD in 
Evaiuation 


Principai 
Survey, F2 


m. 


How to evaluate a core reading program 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items are 
checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


n. 


How to evaluate reading instruction 


0. 


How to coach teachers in reading instruction 


P- 


How to manage reading personnel 
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Construct 


Survey 


Survey 


Survey Description 


Item Scale 


Scoring Metric 




Question 


Item 









Professional Development for Building Administrators (continued) 



Received PD in 
Assessment 


Principal 
Survey, F2 


q- 


How to diagnose reading problems 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items are 
checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


r. 


How to administer assessments 


s. 


How to interpret and use assessment data to guide 
instruction 


Received PD for 
Struggiing Readers 


Principal 
Survey, F2 


t. 


How to help struggling readers with decoding/phonics 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items are 
checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


u. 


How to help struggling readers with vocabulary 


V. 


How to help struggling readers with comprehension 


X. 


Strategies for teaching reading to students with 
diagnosed learning disabilities 


Received PD in 
Organization, 
Management, and 
Support 


Principal 
Survey, F2 


y- 


How to select reading materials 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 if any of 
Survey Items are 
checked “Topic 
addressed in PD.” 

(Range: 0 to 1) 


aa. 


How to select reading assessments 


bb. 


Alignment of reading curriculum and instruction with 
state/district assessments 


dd. 


Classroom management 


Diversity of PD 
Topics Received 


Principal 
Survey, F2 


a - e. 


Received PD in content of reading instruction construct 
(see above) 


Each construct is 
scored as 1 = 
“Received PD” or 
0 = “Did not 
Receive PD” 
except a-e, which is 
scored as 0 to 5 — 
the sum of content 
areas teachers 
“Received PD” 
(individually scored 
as 0 or 1) 


Sum of the above 
constructs (which are 
scored 0-5 or 0-1). 

(Range: 0 to 10) 


f-l. 


Received PD in teaching strategies construct (see 
above) 


m - p. 


Received PD in evaluation construct (see above) 


q-s. 


Received PD in assessment construct (see above) 


t-x. 


Received PD for struggling readers construct (see 
above) 


y-dd. 


Received PD in organization, management, and support 
construct (see above) 
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Construct 


Survey 

Question 


Survey 

Item 


Survey Description 


Item Scale 


Scoring Metric 


Professional Development for Reading Coaches 


Diversity of PD 
Topics Received — 
Reading Coach 
Skiiis 


Reading 
Coach 
Survey, G1 


a. 


How to use reading assessment data to guide 
instruction 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 for each 
Survey Item checked 
“Topic addressed in 
PD.” 

Sum Survey Item 
scores 

(Range: 0 to 8) 


c. 


How to use assessment data to form instructional 
groups 


f. 


Essential components of SBR instruction 


h. 


How to plan instructional interventions for struggling 
readers 


j- 


How to conduct effective grade level meetings 


I. 


How to help teachers make reading instruction 
systematic and explicit 


m. 


How to construct demonstration lessons 


n. 


How to conduct classroom observations 


Diversity of PD 
Topics Received — 
General Coaching 
Skills 


Reading 
Coach 
Survey, G1 


b. 


What are the types of assessments: screening, 
diagnostic, progress, monitoring, and outcome 


Dichotomous 
1 = “Topic 
addressed in 
Professional 
Development” 
0 = “Topic not 
addressed” 


Score 1 for each 
Survey Item checked 
“Topic addressed in 
PD.” 

Sum Survey Item 
scores 

(Range: 0 to 7) 


d. 


How to provide constructive feedback to teachers 


e. 


How to establish credibility with teachers 


g- 


What is the role of the reading coach in fostering change 


i. 


Classroom management within the literacy block time 


k. 


How to help teachers identify appropriate instructional 
methods 


0. 


How to provide onsite professional development 


Reading Coach Acti 


ivities 


Reading Coach 
Activities 


Reading 
Coach 
Survey, B5 


a.-j. 


List of coaching activities 


5 item Likert scale 
1 = “do not do or 
not at all central” 

5 = “absolutely 
central” 


Mean of all (a-j) 
Survey Item scores 
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Construct 


Survey 


Survey 


Survey Description 


Item Scale 


Scoring Metric 
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Item 









Role of Reading Coach 



Reading Coach 
Role — Teacher- 
Support Tasks 


Reading 
Coach 
Survey, B4 


c. 


Facilitating grade level meetings 


5 item Likert scale 
1 = “do not do or 
not at all central” 

5 = “absolutely 
central” 


Sum Survey Item 
scores (1-5 scale) for 
non-distractor items 

Subtract Survey Item 
score for distractor 
(1-5 scale) 

Take mean 


j- 


Providing training/professional development in reading 
materials, strategies, and assessments 


k. 


Coaching staff on a range of topics 


I. 


Organizing professional development for K-3 teachers 


/. 


Providing direct reading instruction to students 


Reading Coach 
Role — 

Administrative 

Tasks 


Reading 
Coach 
Survey, B4 


a. 


Administering/coordinating reading assessments 


5 item Likert scale 
1 = “do not do or 
not at all central” 

5 = “absolutely 
central” 


Mean of Survey Item 
scores 


b. 


Compiling reading assessment data for teachers 


e. 


Facilitating or coordinating family literacy activities 


f. 


Ordering/managing reading instruction materials 


h. 


Providing sub time for teachers to observe other more 
experienced teachers 
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